pwc-archive/papers-with-abstracts huggingface dataset:
Harmonic Recomposition using Conditional Autoregressive Modeling
Authors: Kyle Kastner, Rithesh Kumar, Tim Cooijmans, Aaron Courville
Date: 2018-11-18 00:00:00

We demonstrate a conditional autoregressive pipeline for efficient music
recomposition, based on methods presented in van den Oord et al.(2017).
Recomposition (Casal & Casey, 2010) focuses on reworking existing musical
pieces, adhering to structure at a high level while also re-imagining other
aspects of the work. This can involve reuse of pre-existing themes or parts of
the original piece, while also requiring the flexibility to generate new
content at different levels of granularity. Applying the aforementioned
modeling pipeline to recomposition, we show diverse and structured generation
conditioned on chord sequence annotations.
PerSIM: Multi-resolution Image Quality Assessment in the Perceptually Uniform Color Domain
Authors: Dogancan Temel, Ghassan AlRegib
Date: 2018-11-18 00:00:00
Tasks: Image Quality Assessment

An average observer perceives the world in color instead of black and white.
Moreover, the visual system focuses on structures and segments instead of
individual pixels. Based on these observations, we propose a full reference
objective image quality metric modeling visual system characteristics and
chroma similarity in the perceptually uniform color domain (Lab). Laplacian of
Gaussian features are obtained in the L channel to model the retinal ganglion
cells in human visual system and color similarity is calculated over the a and
b channels. In the proposed perceptual similarity index (PerSIM), a
multi-resolution approach is followed to mimic the hierarchical nature of human
visual system. LIVE and TID2013 databases are used in the validation and PerSIM
outperforms all the compared metrics in the overall databases in terms of
ranking, monotonic behavior and linearity.
Realtime Scheduling and Power Allocation Using Deep Neural Networks
Authors: Shenghe Xu, Pei Liu, Ran Wang, Shivendra S. Panwar
Date: 2018-11-18 00:00:00
Tasks: Management, Scheduling

With the increasing number of base stations (BSs) and network densification
in 5G, interference management using link scheduling and power control are
vital for better utilization of radio resources. However, the complexity of
solving link scheduling and the power control problem grows exponentially with
the number of BS. Due to high computation time, previous methods are useful for
research purposes but impractical for real time usage. In this paper we propose
to use deep neural networks (DNNs) to approximate optimal link scheduling and
power control for the case with multiple small cells. A deep Q-network (DQN)
estimates a suitable schedule, then a DNN allocates power for the corresponding
schedule. Simulation results show that the proposed method achieves over five
orders of magnitude speed-up with less than nine percent performance loss,
making real time usage practical.
Multimodal Densenet
Authors: Faisal Mahmood, Ziyun Yang, Thomas Ashley, Nicholas J. Durr
Date: 2018-11-18 00:00:00
Tasks: Prognosis

Humans make accurate decisions by interpreting complex data from multiple
sources. Medical diagnostics, in particular, often hinge on human
interpretation of multi-modal information. In order for artificial intelligence
to make progress in automated, objective, and accurate diagnosis and prognosis,
methods to fuse information from multiple medical imaging modalities are
required. However, combining information from multiple data sources has several
challenges, as current deep learning architectures lack the ability to extract
useful representations from multimodal information, and often simple
concatenation is used to fuse such information. In this work, we propose
Multimodal DenseNet, a novel architecture for fusing multimodal data. Instead
of focusing on concatenation or early and late fusion, our proposed
architectures fuses information over several layers and gives the model
flexibility in how it combines information from multiple sources. We apply this
architecture to the challenge of polyp characterization and landmark
identification in endoscopy. Features from white light images are fused with
features from narrow band imaging or depth maps. This study demonstrates that
Multimodal DenseNet outperforms monomodal classification as well as other
multimodal fusion techniques by a significant margin on two different datasets.
Facial Expression and Peripheral Physiology Fusion to Decode Individualized Affective Experience
Authors: Yu Yin, Mohsen Nabian, Miolin Fan, Chun-An Chou, Maria Gendron, Sarah Ostadabbas
Date: 2018-11-18 00:00:00
Tasks: Prediction

In this paper, we present a multimodal approach to simultaneously analyze
facial movements and several peripheral physiological signals to decode
individualized affective experiences under positive and negative emotional
contexts, while considering their personalized resting dynamics. We propose a
person-specific recurrence network to quantify the dynamics present in the
person's facial movements and physiological data. Facial movement is
represented using a robust head vs. 3D face landmark localization and tracking
approach, and physiological data are processed by extracting known attributes
related to the underlying affective experience. The dynamical coupling between
different input modalities is then assessed through the extraction of several
complex recurrent network metrics. Inference models are then trained using
these metrics as features to predict individual's affective experience in a
given context, after their resting dynamics are excluded from their response.
We validated our approach using a multimodal dataset consists of (i) facial
videos and (ii) several peripheral physiological signals, synchronously
recorded from 12 participants while watching 4 emotion-eliciting video-based
stimuli. The affective experience prediction results signified that our
multimodal fusion method improves the prediction accuracy up to 19% when
compared to the prediction using only one or a subset of the input modalities.
Furthermore, we gained prediction improvement for affective experience by
considering the effect of individualized resting dynamics.
Temporal Recurrent Networks for Online Action Detection
Authors: Mingze Xu, Mingfei Gao, Yi-Ting Chen, Larry S. Davis, David J. Crandall
Date: 2018-11-18 00:00:00
Proceeding: ICCV 2019 10
Tasks: Action Detection, Online Action Detection

Most work on temporal action detection is formulated as an offline problem,
in which the start and end times of actions are determined after the entire
video is fully observed. However, important real-time applications including
surveillance and driver assistance systems require identifying actions as soon
as each video frame arrives, based only on current and historical observations.
In this paper, we propose a novel framework, Temporal Recurrent Network (TRN),
to model greater temporal context of a video frame by simultaneously performing
online action detection and anticipation of the immediate future. At each
moment in time, our approach makes use of both accumulated historical evidence
and predicted future information to better recognize the action that is
currently occurring, and integrates both of these into a unified end-to-end
architecture. We evaluate our approach on two popular online action detection
datasets, HDD and TVSeries, as well as another widely used dataset, THUMOS'14.
The results show that TRN significantly outperforms the state-of-the-art.
Deep Siamese Networks with Bayesian non-Parametrics for Video Object Tracking
Authors: Anthony D. Rhodes, Manan Goel
Date: 2018-11-18 00:00:00
Tasks: Bayesian Optimization, Object, Object Tracking, Video Object Tracking

We present a novel algorithm utilizing a deep Siamese neural network as a
general object similarity function in combination with a Bayesian optimization
(BO) framework to encode spatio-temporal information for efficient object
tracking in video. In particular, we treat the video tracking problem as a
dynamic (i.e. temporally-evolving) optimization problem. Using Gaussian Process
priors, we model a dynamic objective function representing the location of a
tracked object in each frame. By exploiting temporal correlations, the proposed
method queries the search space in a statistically principled and efficient
way, offering several benefits over current state of the art video tracking
methods.
RGB-based 3D Hand Pose Estimation via Privileged Learning with Depth Images
Authors: Shanxin Yuan, Bjorn Stenger, Tae-Kyun Kim
Date: 2018-11-18 00:00:00
Tasks: 3D Hand Pose Estimation, Hand Pose Estimation, Pose Estimation

This paper proposes a method for hand pose estimation from RGB images that
uses both external large-scale depth image datasets and paired depth and RGB
images as privileged information at training time. We show that providing depth
information during training significantly improves performance of pose
estimation from RGB images during testing. We explore different ways of using
this privileged information: (1) using depth data to initially train a
depth-based network, (2) using the features from the depth-based network of the
paired depth images to constrain mid-level RGB network weights, and (3) using
the foreground mask, obtained from the depth data, to suppress the responses
from the background area. By using paired RGB and depth images, we are able to
supervise the RGB-based network to learn middle layer features that mimic that
of the corresponding depth-based network, which is trained on large-scale,
accurately annotated depth data. During testing, when only an RGB image is
available, our method produces accurate 3D hand pose predictions. Our method is
also tested on 2D hand pose estimation. Experiments on three public datasets
show that the method outperforms the state-of-the-art methods for hand pose
estimation using RGB image input.
The Taboo Trap: Behavioural Detection of Adversarial Samples
Authors: Ilia Shumailov, Yiren Zhao, Robert Mullins, Ross Anderson
Date: 2018-11-18 00:00:00

Deep Neural Networks (DNNs) have become a powerful toolfor a wide range of problems. Yet recent work has found an increasing variety of adversarial samplesthat can fool them. Most existing detection mechanisms against adversarial attacksimpose significant costs, either by using additional classifiers to spot adversarial samples, or by requiring the DNN to be restructured. In this paper, we introduce a novel defence. We train our DNN so that, as long as it is workingas intended on the kind of inputs we expect, its behavior is constrained, in that some set of behaviors are taboo. If it is exposed to adversarial samples, they will often cause a taboo behavior, which we can detect. Taboos can be both subtle and diverse, so their choice can encode and hide information. It is a well-established design principle that the security of a system should not depend on the obscurity of its design, but on some variable (the key) which can differ between implementations and bechanged as necessary. We discuss how taboos can be used to equip a classifier with just such a key, and how to tune the keying mechanism to adversaries of various capabilities. We evaluate the performance of a prototype against a wide range of attacks and show how our simple defense can defend against cheap attacks at scale with zero run-time computation overhead, making it a suitable defense method for IoT devices.
Neural Multi-Task Learning for Citation Function and Provenance
Authors: Xuan Su, Animesh Prasad, Min-Yen Kan, Kazunari Sugiyama
Date: 2018-11-18 00:00:00
Tasks: Multi-Task Learning

Citation function and provenance are two cornerstone tasks in citation
analysis. Given a citation, the former task determines its rhetorical role,
while the latter locates the text in the cited paper that contains the relevant
cited information. We hypothesize that these two tasks are synergistically
related, and build a model that validates this claim. For both tasks, we show
that a single-layer convolutional neural network (CNN) outperforms existing
state-of-the-art baselines. More importantly, we show that the two tasks are
indeed synergistic: by jointly training both of the tasks in a multi-task
learning setup, we demonstrate additional performance gains. Altogether, our
models improve the current state-of-the-arts up to 2\%, with statistical
significance for both citation function and provenance prediction tasks.
Policy Optimization with Model-based Explorations
Authors: Feiyang Pan, Qingpeng Cai, An-Xiang Zeng, Chun-Xiang Pan, Qing Da, Hua-Lin He, Qing He, Pingzhong Tang
Date: 2018-11-18 00:00:00
Tasks: Atari Games, Decision Making, model, Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Model-free reinforcement learning methods such as the Proximal Policy
Optimization algorithm (PPO) have successfully applied in complex
decision-making problems such as Atari games. However, these methods suffer
from high variances and high sample complexity. On the other hand, model-based
reinforcement learning methods that learn the transition dynamics are more
sample efficient, but they often suffer from the bias of the transition
estimation. How to make use of both model-based and model-free learning is a
central problem in reinforcement learning. In this paper, we present a new
technique to address the trade-off between exploration and exploitation, which
regards the difference between model-free and model-based estimations as a
measure of exploration value. We apply this new technique to the PPO algorithm
and arrive at a new policy optimization method, named Policy Optimization with
Model-based Explorations (POME). POME uses two components to predict the
actions' target values: a model-free one estimated by Monte-Carlo sampling and
a model-based one which learns a transition model and predicts the value of the
next state. POME adds the error of these two target estimations as the
additional exploration value for each state-action pair, i.e, encourages the
algorithm to explore the states with larger target errors which are hard to
estimate. We compare POME with PPO on Atari 2600 games, and it shows that POME
outperforms PPO on 33 games out of 49 games.
Transfer Learning with Deep CNNs for Gender Recognition and Age Estimation
Authors: Philip Smith, Cuixian Chen
Date: 2018-11-18 00:00:00
Tasks: Age Estimation, Data Augmentation, Transfer Learning

In this project, competition-winning deep neural networks with pretrained
weights are used for image-based gender recognition and age estimation.
Transfer learning is explored using both VGG19 and VGGFace pretrained models by
testing the effects of changes in various design schemes and training
parameters in order to improve prediction accuracy. Training techniques such as
input standardization, data augmentation, and label distribution age encoding
are compared. Finally, a hierarchy of deep CNNs is tested that first classifies
subjects by gender, and then uses separate male and female age models to
predict age. A gender recognition accuracy of 98.7% and an MAE of 4.1 years is
achieved. This paper shows that, with proper training techniques, good results
can be obtained by retasking existing convolutional filters towards a new
purpose.
Transform-Based Multilinear Dynamical System for Tensor Time Series Analysis
Authors: Weijun Lu, Xiao-Yang Liu, Qingwei Wu, Yue Sun, Anwar Walid
Date: 2018-11-18 00:00:00
Tasks: Time Series, Time Series Analysis

We propose a novel multilinear dynamical system (MLDS) in a transform domain,
named $\mathcal{L}$-MLDS, to model tensor time series. With transformations
applied to a tensor data, the latent multidimensional correlations among the
frontal slices are built, and thus resulting in the computational independence
in the transform domain. This allows the exact separability of the
multi-dimensional problem into multiple smaller LDS problems. To estimate the
system parameters, we utilize the expectation-maximization (EM) algorithm to
determine the parameters of each LDS. Further, $\mathcal{L}$-MLDSs
significantly reduce the model parameters and allows parallel processing. Our
general $\mathcal{L}$-MLDS model is implemented based on different transforms:
discrete Fourier transform, discrete cosine transform and discrete wavelet
transform. Due to the nonlinearity of these transformations, $\mathcal{L}$-MLDS
is able to capture the nonlinear correlations within the data unlike the MLDS
\cite{rogers2013multilinear} which assumes multi-way linear correlations. Using
four real datasets, the proposed $\mathcal{L}$-MLDS is shown to achieve much
higher prediction accuracy than the state-of-the-art MLDS and LDS with an equal
number of parameters under different noise models. In particular, the relative
errors are reduced by $50\% \sim 99\%$. Simultaneously, $\mathcal{L}$-MLDS
achieves an exponential improvement in the model's training time than MLDS.
Implementation of Robust Face Recognition System Using Live Video Feed Based on CNN
Authors: Yang Li, Sangwhan Cha
Date: 2018-11-18 00:00:00
Tasks: Face Recognition, Iris Recognition, Robust Face Recognition

The way to accurately and effectively identify people has always been an
interesting topic in research and industry. With the rapid development of
artificial intelligence in recent years, facial recognition gains lots of
attention due to prompting the development of emerging identification methods.
Compared to traditional card recognition, fingerprint recognition and iris
recognition, face recognition has many advantages including non-contact
interface, high concurrency, and user-friendly usage. It has high potential to
be used in government, public facilities, security, e-commerce, retailing,
education and many other fields. With the development of deep learning and the
introduction of deep convolutional neural networks, the accuracy and speed of
face recognition have made great strides. However, the results from different
networks and models are very different with different system architecture.
Furthermore, it could take significant amount of data storage space and data
processing time for the face recognition system with video feed, if the system
stores images and features of human faces. In this paper, facial features are
extracted by merging and comparing multiple models, and then a deep neural
network is constructed to train and construct the combined features. In this
way, the advantages of multiple models can be combined to mention the
recognition accuracy. After getting a model with high accuracy, we build a
product model. The model will take a human face image and extract it into a
vector. Then the distance between vectors are compared to determine if two
faces on different picture belongs to the same person. The proposed approach
reduces data storage space and data processing time for the face recognition
system with video feed scientifically with our proposed system architecture.
Distribution Discrepancy Maximization for Image Privacy Preserving
Authors: Sen Liu, Jianxin Lin, Zhibo Chen
Date: 2018-11-18 00:00:00
Tasks: Privacy Preserving

With the rapid increase in online photo sharing activities, image obfuscation
algorithms become particularly important for protecting the sensitive
information in the shared photos. However, existing image obfuscation methods
based on hand-crafted principles are challenged by the dramatic development of
deep learning techniques. To address this problem, we propose to maximize the
distribution discrepancy between the original image domain and the encrypted
image domain. Accordingly, we introduce a collaborative training scheme: a
discriminator $D$ is trained to discriminate the reconstructed image from the
encrypted image, and an encryption model $G_e$ is required to generate these
two kinds of images to maximize the recognition rate of $D$, leading to the
same training objective for both $D$ and $G_e$. We theoretically prove that
such a training scheme maximizes two distributions' discrepancy. Compared with
commonly-used image obfuscation methods, our model can produce satisfactory
defense against the attack of deep recognition models indicated by significant
accuracy decreases on FaceScrub, Casia-WebFace and LFW datasets.
On Matching Faces with Alterations due to Plastic Surgery and Disguise
Authors: Saksham Suri, Anush Sankaran, Mayank Vatsa, Richa Singh
Date: 2018-11-18 00:00:00
Tasks: Face Recognition

Plastic surgery and disguise variations are two of the most challenging
co-variates of face recognition. The state-of-art deep learning models are not
sufficiently successful due to the availability of limited training samples. In
this paper, a novel framework is proposed which transfers fundamental visual
features learnt from a generic image dataset to supplement a supervised face
recognition model. The proposed algorithm combines off-the-shelf supervised
classifier and a generic, task independent network which encodes information
related to basic visual cues such as color, shape, and texture. Experiments are
performed on IIITD plastic surgery face dataset and Disguised Faces in the Wild
(DFW) dataset. Results showcase that the proposed algorithm achieves state of
the art results on both the datasets. Specifically on the DFW database, the
proposed algorithm yields over 87% verification accuracy at 1% false accept
rate which is 53.8% better than baseline results computed using VGGFace.
Learning to infer: RL-based search for DNN primitive selection on Heterogeneous Embedded Systems
Authors: Miguel de Prado, Nuria Pazos, Luca Benini
Date: 2018-11-18 00:00:00
Tasks: CPU, Reinforcement Learning

Deep Learning is increasingly being adopted by industry for computer vision
applications running on embedded devices. While Convolutional Neural Networks'
accuracy has achieved a mature and remarkable state, inference latency and
throughput are a major concern especially when targeting low-cost and low-power
embedded platforms. CNNs' inference latency may become a bottleneck for Deep
Learning adoption by industry, as it is a crucial specification for many
real-time processes. Furthermore, deployment of CNNs across heterogeneous
platforms presents major compatibility issues due to vendor-specific technology
and acceleration libraries. In this work, we present QS-DNN, a fully automatic
search based on Reinforcement Learning which, combined with an inference engine
optimizer, efficiently explores through the design space and empirically finds
the optimal combinations of libraries and primitives to speed up the inference
of CNNs on heterogeneous embedded devices. We show that, an optimized
combination can achieve 45x speedup in inference latency on CPU compared to a
dependency-free baseline and 2x on average on GPGPU compared to the best vendor
library. Further, we demonstrate that, the quality of results and time
"to-solution" is much better than with Random Search and achieves up to 15x
better results for a short-time search.