pwc-archive/papers-with-abstracts huggingface dataset:
Normalization of Neural Networks using Analytic Variance Propagation
Authors: Alexander Shekhovtsov, Boris Flach
Date: 2018-03-28 00:00:00

We address the problem of estimating statistics of hidden units in a neural
network using a method of analytic moment propagation. These statistics are
useful for approximate whitening of the inputs in front of saturating
non-linearities such as a sigmoid function. This is important for
initialization of training and for reducing the accumulated scale and bias
dependencies (compensating covariate shift), which presumably eases the
learning. In batch normalization, which is currently a very widely applied
technique, sample estimates of statistics of hidden units over a batch are
used. The proposed estimation uses an analytic propagation of mean and variance
of the training set through the network. The result depends on the network
structure and its current weights but not on the specific batch input. The
estimates are suitable for initialization and normalization, efficient to
compute and independent of the batch size. The experimental verification well
supports these claims. However, the method does not share the generalization
properties of BN, to which our experiments give some additional insight.
Joint PLDA for Simultaneous Modeling of Two Factors
Authors: Luciana Ferrer, Mitchell McLaren
Date: 2018-03-28 00:00:00
Tasks: Face Recognition, Speaker Verification, Vocal Bursts Valence Prediction

Probabilistic linear discriminant analysis (PLDA) is a method used for
biometric problems like speaker or face recognition that models the variability
of the samples using two latent variables, one that depends on the class of the
sample and another one that is assumed independent across samples and models
the within-class variability. In this work, we propose a generalization of PLDA
that enables joint modeling of two sample-dependent factors: the class of
interest and a nuisance condition. The approach does not change the basic form
of PLDA but rather modifies the training procedure to consider the dependency
across samples of the latent variable that models within-class variability.
While the identity of the nuisance condition is needed during training, it is
not needed during testing since we propose a scoring procedure that
marginalizes over the corresponding latent variable. We show results on a
multilingual speaker-verification task, where the language spoken is considered
a nuisance condition. We show that the proposed joint PLDA approach leads to
significant performance gains in this task for two different datasets, in
particular when the training data contains mostly or only monolingual speakers.
Every Smile is Unique: Landmark-Guided Diverse Smile Generation
Authors: Wei Wang, Xavier Alameda-Pineda, Dan Xu, Pascal Fua, Elisa Ricci, Nicu Sebe
Date: 2018-02-06 00:00:00
Proceeding: CVPR 2018 6
Tasks: Video Generation

Each smile is unique: one person surely smiles in different ways (e.g.,
closing/opening the eyes or mouth). Given one input image of a neutral face,
can we generate multiple smile videos with distinctive characteristics? To
tackle this one-to-many video generation problem, we propose a novel deep
learning architecture named Conditional Multi-Mode Network (CMM-Net). To better
encode the dynamics of facial expressions, CMM-Net explicitly exploits facial
landmarks for generating smile sequences. Specifically, a variational
auto-encoder is used to learn a facial landmark embedding. This single
embedding is then exploited by a conditional recurrent network which generates
a landmark embedding sequence conditioned on a specific expression (e.g.,
spontaneous smile). Next, the generated landmark embeddings are fed into a
multi-mode recurrent landmark generator, producing a set of landmark sequences
still associated to the given smile class but clearly distinct from each other.
Finally, these landmark sequences are translated into face videos. Our
experimental results demonstrate the effectiveness of our CMM-Net in generating
realistic videos of multiple smile expressions.
Context-aware Deep Feature Compression for High-speed Visual Tracking
Authors: Jongwon Choi, Hyung Jin Chang, Tobias Fischer, Sangdoo Yun, Kyuewang Lee, Jiyeoup Jeong, Yiannis Demiris, Jin Young Choi
Date: 2018-03-28 00:00:00
Proceeding: CVPR 2018 6
Tasks: Denoising, Feature Compression, Visual Object Tracking, Visual Tracking, Vocal Bursts Intensity Prediction

We propose a new context-aware correlation filter based tracking framework to
achieve both high computational speed and state-of-the-art performance among
real-time trackers. The major contribution to the high computational speed lies
in the proposed deep feature compression that is achieved by a context-aware
scheme utilizing multiple expert auto-encoders; a context in our framework
refers to the coarse category of the tracking target according to appearance
patterns. In the pre-training phase, one expert auto-encoder is trained per
category. In the tracking phase, the best expert auto-encoder is selected for a
given target, and only this auto-encoder is used. To achieve high tracking
performance with the compressed feature map, we introduce extrinsic denoising
processes and a new orthogonality loss term for pre-training and fine-tuning of
the expert auto-encoders. We validate the proposed context-aware framework
through a number of experiments, where our method achieves a comparable
performance to state-of-the-art trackers which cannot run in real-time, while
running at a significantly fast speed of over 100 fps.
Machine Speech Chain with One-shot Speaker Adaptation
Authors: Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Date: 2018-03-28 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Speaker Recognition, speech-recognition, Speech Recognition, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

In previous work, we developed a closed-loop speech chain model based on deep
learning, in which the architecture enabled the automatic speech recognition
(ASR) and text-to-speech synthesis (TTS) components to mutually improve their
performance. This was accomplished by the two parts teaching each other using
both labeled and unlabeled data. This approach could significantly improve
model performance within a single-speaker speech dataset, but only a slight
increase could be gained in multi-speaker tasks. Furthermore, the model is
still unable to handle unseen speakers. In this paper, we present a new speech
chain mechanism by integrating a speaker recognition model inside the loop. We
also propose extending the capability of TTS to handle unseen speakers by
implementing one-shot speaker adaptation. This enables TTS to mimic voice
characteristics from one speaker to another with only a one-shot speaker
sample, even from a text without any speaker information. In the speech chain
loop mechanism, ASR also benefits from the ability to further learn an
arbitrary speaker's characteristics from the generated speech waveform,
resulting in a significant improvement in the recognition rate.
Quantum algorithms for training Gaussian Processes
Authors: Zhikuan Zhao, Jack K. Fitzsimons, Michael A. Osborne, Stephen J. Roberts, Joseph F. Fitzsimons
Date: 2018-03-28 00:00:00
Tasks: Gaussian Processes

Gaussian processes (GPs) are important models in supervised machine learning.
Training in Gaussian processes refers to selecting the covariance functions and
the associated parameters in order to improve the outcome of predictions, the
core of which amounts to evaluating the logarithm of the marginal likelihood
(LML) of a given model. LML gives a concrete measure of the quality of
prediction that a GP model is expected to achieve. The classical computation of
LML typically carries a polynomial time overhead with respect to the input
size. We propose a quantum algorithm that computes the logarithm of the
determinant of a Hermitian matrix, which runs in logarithmic time for sparse
matrices. This is applied in conjunction with a variant of the quantum linear
system algorithm that allows for logarithmic time computation of the form
$\mathbf{y}^TA^{-1}\mathbf{y}$, where $\mathbf{y}$ is a dense vector and $A$ is
the covariance matrix. We hence show that quantum computing can be used to
estimate the LML of a GP with exponentially improved efficiency under certain
conditions.
Disentangling Factors of Variation by Mixing Them
Authors: Qiyang Hu, Attila Szabó, Tiziano Portenier, Matthias Zwicker, Paolo Favaro
Date: 2017-11-20 00:00:00
Proceeding: CVPR 2018 6
Tasks: Attribute, General Classification, Retrieval

We propose an approach to learn image representations that consist of
disentangled factors of variation without exploiting any manual labeling or
data domain knowledge. A factor of variation corresponds to an image attribute
that can be discerned consistently across a set of images, such as the pose or
color of objects. Our disentangled representation consists of a concatenation
of feature chunks, each chunk representing a factor of variation. It supports
applications such as transferring attributes from one image to another, by
simply mixing and unmixing feature chunks, and classification or retrieval
based on one or several attributes, by considering a user-specified subset of
feature chunks. We learn our representation without any labeling or knowledge
of the data domain, using an autoencoder architecture with two novel training
objectives: first, we propose an invariance objective to encourage that
encoding of each attribute, and decoding of each chunk, are invariant to
changes in other attributes and chunks, respectively; second, we include a
classification objective, which ensures that each chunk corresponds to a
consistently discernible attribute in the represented image, hence avoiding
degenerate feature mappings where some chunks are completely ignored. We
demonstrate the effectiveness of our approach on the MNIST, Sprites, and CelebA
datasets.
Adversarial Spatio-Temporal Learning for Video Deblurring
Authors: Kaihao Zhang, Wenhan Luo, Yiran Zhong, Lin Ma, Wei Liu, Hongdong Li
Date: 2018-03-28 00:00:00
Tasks: Deblurring, Generative Adversarial Network, Video Deblurring

Camera shake or target movement often leads to undesired blur effects in
videos captured by a hand-held camera. Despite significant efforts having been
devoted to video-deblur research, two major challenges remain: 1) how to model
the spatio-temporal characteristics across both the spatial domain (i.e., image
plane) and temporal domain (i.e., neighboring frames), and 2) how to restore
sharp image details w.r.t. the conventionally adopted metric of pixel-wise
errors. In this paper, to address the first challenge, we propose a DeBLuRring
Network (DBLRNet) for spatial-temporal learning by applying a modified 3D
convolution to both spatial and temporal domains. Our DBLRNet is able to
capture jointly spatial and temporal information encoded in neighboring frames,
which directly contributes to improved video deblur performance. To tackle the
second challenge, we leverage the developed DBLRNet as a generator in the GAN
(generative adversarial network) architecture, and employ a content loss in
addition to an adversarial loss for efficient adversarial training. The
developed network, which we name as DeBLuRring Generative Adversarial Network
(DBLRGAN), is tested on two standard benchmarks and achieves the
state-of-the-art performance.
Bundled fragments of first-order modal logic: (un)decidability
Authors: Anantha Padmanabha, R. Ramanujam, Yanjing Wang
Date: 2018-03-28 00:00:00

Quantified modal logic provides a natural logical language for reasoning
about modal attitudes even while retaining the richness of quantification for
referring to predicates over domains. But then most fragments of the logic are
undecidable, over many model classes. Over the years, only a few fragments
(such as the monodic) have been shown to be decidable. In this paper, we study
fragments that bundle quantifiers and modalities together, inspired by earlier
work on epistemic logics of know-how/why/what. As always with quantified modal
logics, it makes a significant difference whether the domain stays the same
across worlds, or not. In particular, we show that the bundle $\forall \Box$ is
undecidable over constant domain interpretations, even with only monadic
predicates, whereas $\exists \Box$ bundle is decidable. On the other hand, over
increasing domain interpretations, we get decidability with both $\forall \Box$
and $\exists \Box$ bundles with unrestricted predicates. In these cases, we
also obtain tableau based procedures that run in \PSPACE. We further show that
the $\exists \Box$ bundle cannot distinguish between constant domain and
increasing domain interpretations.
Fast and Accurate Reconstruction of Compressed Color Light Field
Authors: Ofir Nabati, David Mendlovic, Raja Giryes
Date: 2018-01-31 00:00:00
Tasks: Efficient Neural Network

Light field photography has been studied thoroughly in recent years. One of
its drawbacks is the need for multi-lens in the imaging. To compensate that,
compressed light field photography has been proposed to tackle the trade-offs
between the spatial and angular resolutions. It obtains by only one lens, a
compressed version of the regular multi-lens system. The acquisition system
consists of a dedicated hardware followed by a decompression algorithm, which
usually suffers from high computational time. In this work, we propose a
computationally efficient neural network that recovers a high-quality color
light field from a single coded image. Unlike previous works, we compress the
color channels as well, removing the need for a CFA in the imaging system. Our
approach outperforms existing solutions in terms of recovery quality and
computational complexity. We propose also a neural network for depth map
extraction based on the decompressed light field, which is trained in an
unsupervised manner without the ground truth depth map.
COCO-Stuff: Thing and Stuff Classes in Context
Authors: Holger Caesar, Jasper Uijlings, Vittorio Ferrari
Date: 2016-12-12 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Captioning, Semantic Segmentation, Superpixels

Semantic classes can be either things (objects with a well-defined shape,
e.g. car, person) or stuff (amorphous background regions, e.g. grass, sky).
While lots of classification and detection works focus on thing classes, less
attention has been given to stuff classes. Nonetheless, stuff classes are
important as they allow to explain important aspects of an image, including (1)
scene type; (2) which thing classes are likely to be present and their location
(through contextual reasoning); (3) physical attributes, material types and
geometric properties of the scene. To understand stuff and things in context we
introduce COCO-Stuff, which augments all 164K images of the COCO 2017 dataset
with pixel-wise annotations for 91 stuff classes. We introduce an efficient
stuff annotation protocol based on superpixels, which leverages the original
thing annotations. We quantify the speed versus quality trade-off of our
protocol and explore the relation between annotation time and boundary
complexity. Furthermore, we use COCO-Stuff to analyze: (a) the importance of
stuff and thing classes in terms of their surface cover and how frequently they
are mentioned in image captions; (b) the spatial relations between stuff and
things, highlighting the rich contextual relations that make our dataset
unique; (c) the performance of a modern semantic segmentation method on stuff
and thing classes, and whether stuff is easier to segment than things.
NAG: Network for Adversary Generation
Authors: Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, R. Venkatesh Babu
Date: 2017-12-09 00:00:00
Proceeding: CVPR 2018 6

Adversarial perturbations can pose a serious threat for deploying machine
learning systems. Recent works have shown existence of image-agnostic
perturbations that can fool classifiers over most natural images. Existing
methods present optimization approaches that solve for a fooling objective with
an imperceptibility constraint to craft the perturbations. However, for a given
classifier, they generate one perturbation at a time, which is a single
instance from the manifold of adversarial perturbations. Also, in order to
build robust models, it is essential to explore the manifold of adversarial
perturbations. In this paper, we propose for the first time, a generative
approach to model the distribution of adversarial perturbations. The
architecture of the proposed model is inspired from that of GANs and is trained
using fooling and diversity objectives. Our trained generator network attempts
to capture the distribution of adversarial perturbations for a given classifier
and readily generates a wide variety of such perturbations. Our experimental
evaluation demonstrates that perturbations crafted by our model (i) achieve
state-of-the-art fooling rates, (ii) exhibit wide variety and (iii) deliver
excellent cross model generalizability. Our work can be deemed as an important
step in the process of inferring about the complex manifolds of adversarial
perturbations.
A Factoid Question Answering System for Vietnamese
Authors: Phuong Le-Hong, Duc-Thien Bui
Date: 2018-03-02 00:00:00
Tasks: General Knowledge, Question Answering

In this paper, we describe the development of an end-to-end factoid question
answering system for the Vietnamese language. This system combines both
statistical models and ontology-based methods in a chain of processing modules
to provide high-quality mappings from natural language text to entities. We
present the challenges in the development of such an intelligent user interface
for an isolating language like Vietnamese and show that techniques developed
for inflectional languages cannot be applied "as is". Our question answering
system can answer a wide range of general knowledge questions with promising
accuracy on a test set.
Objects Localisation from Motion with Constraints
Authors: Paul Gay, Alessio Del Bue
Date: 2018-03-28 00:00:00
Tasks: Object, valid

This paper presents a method to estimate the 3D object position and occupancy
given a set of object detections in multiple images and calibrated cameras.
This problem is modelled as the estimation of a set of quadrics given 2D conics
fit to the object bounding boxes. Although a closed form solution has been
recently proposed, the resulting quadrics can be inaccurate or even be non
valid ellipsoids in presence of noisy and inaccurate detections. This effect is
especially important in case of small baselines, resulting in dramatic
failures. To cope with this problem, we propose a set of linear constraints by
matching the centres of the reprojected quadrics with the centres of the
observed conics. These constraints can be solved with a linear system thus
providing a more computationally efficient solution with respect to a
non-linear alternative. Experiments on real data show that the proposed
approach improves significantly the accuracy and the validity of the
ellipsoids.
Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs
Authors: Loic Landrieu, Martin Simonovsky
Date: 2017-11-27 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Semantic Segmentation, Semantic Segmentation

We propose a novel deep learning-based framework to tackle the challenge of
semantic segmentation of large-scale point clouds of millions of points. We
argue that the organization of 3D point clouds can be efficiently captured by a
structure called superpoint graph (SPG), derived from a partition of the
scanned scene into geometrically homogeneous elements. SPGs offer a compact yet
rich representation of contextual relationships between object parts, which is
then exploited by a graph convolutional network. Our framework sets a new state
of the art for segmenting outdoor LiDAR scans (+11.9 and +8.8 mIoU points for
both Semantic3D test sets), as well as indoor scans (+12.4 mIoU points for the
S3DIS dataset).
What deep learning can tell us about higher cognitive functions like mindreading?
Authors: Jaan Aru, Raul Vicente
Date: 2018-03-28 00:00:00
Tasks: Object Recognition

Can deep learning (DL) guide our understanding of computations happening in biological brain? We will first briefly consider how DL has contributed to the research on visual object recognition. In the main part we will assess whether DL could also help us to clarify the computations underlying higher cognitive functions such as Theory of Mind. In addition, we will compare the objectives and learning signals of brains and machines, leading us to conclude that simply scaling up the current DL algorithms will most likely not lead to human level Theory of Mind.
Deep Convolutional Framelet Denosing for Low-Dose CT via Wavelet Residual Network
Authors: Eunhee Kang, Jaejun Yoo, Jong Chul Ye
Date: 2017-07-31 00:00:00
Tasks: Denoising

Model based iterative reconstruction (MBIR) algorithms for low-dose X-ray CT
are computationally expensive. To address this problem, we recently proposed a
deep convolutional neural network (CNN) for low-dose X-ray CT and won the
second place in 2016 AAPM Low-Dose CT Grand Challenge. However, some of the
texture were not fully recovered. To address this problem, here we propose a
novel framelet-based denoising algorithm using wavelet residual network which
synergistically combines the expressive power of deep learning and the
performance guarantee from the framelet-based denoising algorithms. The new
algorithms were inspired by the recent interpretation of the deep convolutional
neural network (CNN) as a cascaded convolution framelet signal representation.
Extensive experimental results confirm that the proposed networks have
significantly improved performance and preserves the detail texture of the
original images.
Clipping free attacks against artificial neural networks
Authors: Boussad ADDAD, Jerome Kodjabachian, Christophe Meyer
Date: 2018-03-26 00:00:00
Tasks: Malware Detection, speech-recognition, Speech Recognition

During the last years, a remarkable breakthrough has been made in AI domain
thanks to artificial deep neural networks that achieved a great success in many
machine learning tasks in computer vision, natural language processing, speech
recognition, malware detection and so on. However, they are highly vulnerable
to easily crafted adversarial examples. Many investigations have pointed out
this fact and different approaches have been proposed to generate attacks while
adding a limited perturbation to the original data. The most robust known
method so far is the so called C&W attack [1]. Nonetheless, a countermeasure
known as feature squeezing coupled with ensemble defense showed that most of
these attacks can be destroyed [6]. In this paper, we present a new method we
call Centered Initial Attack (CIA) whose advantage is twofold : first, it
insures by construction the maximum perturbation to be smaller than a threshold
fixed beforehand, without the clipping process that degrades the quality of
attacks. Second, it is robust against recently introduced defenses such as
feature squeezing, JPEG encoding and even against a voting ensemble of
defenses. While its application is not limited to images, we illustrate this
using five of the current best classifiers on ImageNet dataset among which two
are adversarialy retrained on purpose to be robust against attacks. With a
fixed maximum perturbation of only 1.5% on any pixel, around 80% of attacks
(targeted) fool the voting ensemble defense and nearly 100% when the
perturbation is only 6%. While this shows how it is difficult to defend against
CIA attacks, the last section of the paper gives some guidelines to limit their
impact.