pwc-archive/papers-with-abstracts huggingface dataset:
Visual Attribute Transfer through Deep Image Analogy
Authors: Jing Liao, Yuan YAO, Lu Yuan, Gang Hua, Sing Bing Kang
Date: 2017-05-02 00:00:00
Tasks: Attribute

We propose a new technique for visual attribute transfer across images that
may have very different appearance but have perceptually similar semantic
structure. By visual attribute transfer, we mean transfer of visual information
(such as color, tone, texture, and style) from one image to another. For
example, one image could be that of a painting or a sketch while the other is a
photo of a real scene, and both depict the same type of scene.
  Our technique finds semantically-meaningful dense correspondences between two
input images. To accomplish this, it adapts the notion of "image analogy" with
features extracted from a Deep Convolutional Neutral Network for matching; we
call our technique Deep Image Analogy. A coarse-to-fine strategy is used to
compute the nearest-neighbor field for generating the results. We validate the
effectiveness of our proposed method in a variety of cases, including
style/texture transfer, color/style swap, sketch/painting to photo, and time
lapse.
GAN and VAE from an Optimal Transport Point of View
Authors: Aude Genevay, Gabriel Peyré, Marco Cuturi
Date: 2017-06-06 00:00:00

This short article revisits some of the ideas introduced in arXiv:1701.07875
and arXiv:1705.07642 in a simple setup. This sheds some lights on the
connexions between Variational Autoencoders (VAE), Generative Adversarial
Networks (GAN) and Minimum Kantorovitch Estimators (MKE).
Multi-View Kernels for Low-Dimensional Modeling of Seismic Events
Authors: Ofir Lindenbaum, Yuri Bregman, Neta Rabin, Amir Averbuch
Date: 2017-06-06 00:00:00
Tasks: Dimensionality Reduction

The problem of learning from seismic recordings has been studied for years.
There is a growing interest in developing automatic mechanisms for identifying
the properties of a seismic event. One main motivation is the ability have a
reliable identification of man-made explosions. The availability of multiple
high-dimensional observations has increased the use of machine learning
techniques in a variety of fields. In this work, we propose to use a
kernel-fusion based dimensionality reduction framework for generating
meaningful seismic representations from raw data. The proposed method is tested
on 2023 events that were recorded in Israel and in Jordan. The method achieves
promising results in classification of event type as well as in estimating the
location of the event. The proposed fusion and dimensionality reduction tools
may be applied to other types of geophysical data.
Label-Dependencies Aware Recurrent Neural Networks
Authors: Yoann Dupont, Marco Dinarelli, Isabelle Tellier
Date: 2017-06-06 00:00:00
Tasks: Spoken Language Understanding

In the last few years, Recurrent Neural Networks (RNNs) have proved effective
on several NLP tasks. Despite such great success, their ability to model
\emph{sequence labeling} is still limited. This lead research toward solutions
where RNNs are combined with models which already proved effective in this
domain, such as CRFs. In this work we propose a solution far simpler but very
effective: an evolution of the simple Jordan RNN, where labels are re-injected
as input into the network, and converted into embeddings, in the same way as
words. We compare this RNN variant to all the other RNN models, Elman and
Jordan RNN, LSTM and GRU, on two well-known tasks of Spoken Language
Understanding (SLU). Thanks to label embeddings and their combination at the
hidden layer, the proposed variant, which uses more parameters than Elman and
Jordan RNNs, but far fewer than LSTM and GRU, is more effective than other
RNNs, but also outperforms sophisticated CRF models.
Early Stopping without a Validation Set
Authors: Maren Mahsereci, Lukas Balles, Christoph Lassner, Philipp Hennig
Date: 2017-03-28 00:00:00
Tasks: regression

Early stopping is a widely used technique to prevent poor generalization
performance when training an over-expressive model by means of gradient-based
optimization. To find a good point to halt the optimizer, a common practice is
to split the dataset into a training and a smaller validation set to obtain an
ongoing estimate of the generalization performance. We propose a novel early
stopping criterion based on fast-to-compute local statistics of the computed
gradients and entirely removes the need for a held-out validation set. Our
experiments show that this is a viable approach in the setting of least-squares
and logistic regression, as well as neural networks.
Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC
Authors: Yulai Cong, Bo Chen, Hongwei Liu, Mingyuan Zhou
Date: 2017-06-06 00:00:00
Proceeding: ICML 2017 8
Tasks: Data Augmentation

It is challenging to develop stochastic gradient based scalable inference for
deep discrete latent variable models (LVMs), due to the difficulties in not
only computing the gradients, but also adapting the step sizes to different
latent factors and hidden layers. For the Poisson gamma belief network (PGBN),
a recently proposed deep discrete LVM, we derive an alternative representation
that is referred to as deep latent Dirichlet allocation (DLDA). Exploiting data
augmentation and marginalization techniques, we derive a block-diagonal Fisher
information matrix and its inverse for the simplex-constrained global model
parameters of DLDA. Exploiting that Fisher information matrix with stochastic
gradient MCMC, we present topic-layer-adaptive stochastic gradient Riemannian
(TLASGR) MCMC that jointly learns simplex-constrained global parameters across
all layers and topics, with topic and layer specific learning rates.
State-of-the-art results are demonstrated on big data sets.
A General-Purpose Tagger with Convolutional Neural Networks
Authors: Xiang Yu, Agnieszka Faleńska, Ngoc Thang Vu
Date: 2017-06-06 00:00:00
Proceeding: WS 2017 9
Tasks: Morphological Tagging, Part-Of-Speech Tagging

We present a general-purpose tagger based on convolutional neural networks
(CNN), used for both composing word vectors and encoding context information.
The CNN tagger is robust across different tagging tasks: without task-specific
tuning of hyper-parameters, it achieves state-of-the-art results in
part-of-speech tagging, morphological tagging and supertagging. The CNN tagger
is also robust against the out-of-vocabulary problem, it performs well on
artificially unnormalized texts.
Borrowing Treasures from the Wealthy: Deep Transfer Learning through Selective Joint Fine-tuning
Authors: Weifeng Ge, Yizhou Yu
Date: 2017-02-28 00:00:00
Proceeding: CVPR 2017 7
Tasks: General Classification, Transfer Learning

Deep neural networks require a large amount of labeled training data during
supervised learning. However, collecting and labeling so much data might be
infeasible in many cases. In this paper, we introduce a source-target selective
joint fine-tuning scheme for improving the performance of deep learning tasks
with insufficient training data. In this scheme, a target learning task with
insufficient training data is carried out simultaneously with another source
learning task with abundant training data. However, the source learning task
does not use all existing training data. Our core idea is to identify and use a
subset of training images from the original source learning task whose
low-level characteristics are similar to those from the target learning task,
and jointly fine-tune shared convolutional layers for both tasks. Specifically,
we compute descriptors from linear or nonlinear filter bank responses on
training images from both tasks, and use such descriptors to search for a
desired subset of training samples for the source learning task.
  Experiments demonstrate that our selective joint fine-tuning scheme achieves
state-of-the-art performance on multiple visual classification tasks with
insufficient training data for deep learning. Such tasks include Caltech 256,
MIT Indoor 67, Oxford Flowers 102 and Stanford Dogs 120. In comparison to
fine-tuning without a source domain, the proposed method can improve the
classification accuracy by 2% - 10% using a single model.
Fast rates for online learning in Linearly Solvable Markov Decision Processes
Authors: Gergely Neu, Vicenç Gómez
Date: 2017-02-21 00:00:00

We study the problem of online learning in a class of Markov decision
processes known as linearly solvable MDPs. In the stationary version of this
problem, a learner interacts with its environment by directly controlling the
state transitions, attempting to balance a fixed state-dependent cost and a
certain smooth cost penalizing extreme control inputs. In the current paper, we
consider an online setting where the state costs may change arbitrarily between
consecutive rounds, and the learner only observes the costs at the end of each
respective round. We are interested in constructing algorithms for the learner
that guarantee small regret against the best stationary control policy chosen
in full knowledge of the cost sequence. Our main result is showing that the
smoothness of the control cost enables the simple algorithm of following the
leader to achieve a regret of order $\log^2 T$ after $T$ rounds, vastly
improving on the best known regret bound of order $T^{3/4}$ for this setting.
A Frame Tracking Model for Memory-Enhanced Dialogue Systems
Authors: Hannes Schulz, Jeremie Zumer, Layla El Asri, Shikhar Sharma
Date: 2017-06-06 00:00:00
Proceeding: WS 2017 8

Recently, resources and tasks were proposed to go beyond state tracking in
dialogue systems. An example is the frame tracking task, which requires
recording multiple frames, one for each user goal set during the dialogue. This
allows a user, for instance, to compare items corresponding to different goals.
This paper proposes a model which takes as input the list of frames created so
far during the dialogue, the current user utterance as well as the dialogue
acts, slot types, and slot values associated with this utterance. The model
then outputs the frame being referenced by each triple of dialogue act, slot
type, and slot value. We show that on the recently published Frames dataset,
this model significantly outperforms a previously proposed rule-based baseline.
In addition, we propose an extensive analysis of the frame tracking task by
dividing it into sub-tasks and assessing their difficulty with respect to our
model.
Empirical Bayes Matrix Completion
Authors: Takeru Matsuda, Fumiyasu Komaki
Date: 2017-06-05 00:00:00
Tasks: Matrix Completion

We develop an empirical Bayes (EB) algorithm for the matrix completion
problems. The EB algorithm is motivated from the singular value shrinkage
estimator for matrix means by Efron and Morris (1972). Since the EB algorithm
is essentially the EM algorithm applied to a simple model, it does not require
heuristic parameter tuning other than tolerance. Numerical results demonstrated
that the EB algorithm achieves a good trade-off between accuracy and efficiency
compared to existing algorithms and that it works particularly well when the
difference between the number of rows and columns is large. Application to real
data also shows the practical utility of the EB algorithm.
Compression Fractures Detection on CT
Authors: Amir Bar, Lior Wolf, Orna Bergman Amitai, Eyal Toledano, Eldad Elnekave
Date: 2017-06-06 00:00:00
Tasks: Computed Tomography (CT)

The presence of a vertebral compression fracture is highly indicative of
osteoporosis and represents the single most robust predictor for development of
a second osteoporotic fracture in the spine or elsewhere. Less than one third
of vertebral compression fractures are diagnosed clinically. We present an
automated method for detecting spine compression fractures in Computed
Tomography (CT) scans. The algorithm is composed of three processes. First, the
spinal column is segmented and sagittal patches are extracted. The patches are
then binary classified using a Convolutional Neural Network (CNN). Finally a
Recurrent Neural Network (RNN) is utilized to predict whether a vertebral
fracture is present in the series of patches.
Learning Pairwise Disjoint Simple Languages from Positive Examples
Authors: Alexis Linard, Rick Smetsers, Frits Vaandrager, Umar Waqas, Joost van Pinxten, Sicco Verwer
Date: 2017-06-06 00:00:00
Tasks: Clustering

A classical problem in grammatical inference is to identify a deterministic
finite automaton (DFA) from a set of positive and negative examples. In this
paper, we address the related - yet seemingly novel - problem of identifying a
set of DFAs from examples that belong to different unknown simple regular
languages. We propose two methods based on compression for clustering the
observed positive examples. We apply our methods to a set of print jobs
submitted to large industrial printers.
A Minimal Solution for Two-view Focal-length Estimation using Two Affine Correspondences
Authors: Daniel Barath, Tekla Toth, Levente Hajder
Date: 2017-06-06 00:00:00
Proceeding: CVPR 2017 7
Tasks: Vocal Bursts Valence Prediction

A minimal solution using two affine correspondences is presented to estimate
the common focal length and the fundamental matrix between two semi-calibrated
cameras - known intrinsic parameters except a common focal length. To the best
of our knowledge, this problem is unsolved. The proposed approach extends point
correspondence-based techniques with linear constraints derived from local
affine transformations. The obtained multivariate polynomial system is
efficiently solved by the hidden-variable technique. Observing the geometry of
local affinities, we introduce novel conditions eliminating invalid roots. To
select the best one out of the remaining candidates, a root selection technique
is proposed outperforming the recent ones especially in case of high-level
noise. The proposed 2-point algorithm is validated on both synthetic data and
104 publicly available real image pairs. A Matlab implementation of the
proposed solution is included in the paper.
Volume Calculation of CT lung Lesions based on Halton Low-discrepancy Sequences
Authors: Liansheng Wang, Shusheng Li, Shuo Li
Date: 2017-06-06 00:00:00
Tasks: Computed Tomography (CT)

Volume calculation from the Computed Tomography (CT) lung lesions data is a
significant parameter for clinical diagnosis. The volume is widely used to
assess the severity of the lung nodules and track its progression, however, the
accuracy and efficiency of previous studies are not well achieved for clinical
uses. It remains to be a challenging task due to its tight attachment to the
lung wall, inhomogeneous background noises and large variations in sizes and
shape. In this paper, we employ Halton low-discrepancy sequences to calculate
the volume of the lung lesions. The proposed method directly compute the volume
without the procedure of three-dimension (3D) model reconstruction and surface
triangulation, which significantly improves the efficiency and reduces the
complexity. The main steps of the proposed method are: (1) generate a certain
number of random points in each slice using Halton low-discrepancy sequences
and calculate the lesion area of each slice through the proportion; (2) obtain
the volume by integrating the areas in the sagittal direction. In order to
evaluate our proposed method, the experiments were conducted on the sufficient
data sets with different size of lung lesions. With the uniform distribution of
random points, our proposed method achieves more accurate results compared with
other methods, which demonstrates the robustness and accuracy for the volume
calculation of CT lung lesions. In addition, our proposed method is easy to
follow and can be extensively applied to other applications, e.g., volume
calculation of liver tumor, atrial wall aneurysm, etc.
Retrosynthetic reaction prediction using neural sequence-to-sequence models
Authors: Bowen Liu, Bharath Ramsundar, Prasad Kawthekar, Jade Shi, Joseph Gomes, Quang Luu Nguyen, Stephen Ho, Jack Sloane, Paul Wender, Vijay Pande
Date: 2017-06-06 00:00:00
Tasks: Decoder, Machine Translation, Prediction, Single-step retrosynthesis, Translation

We describe a fully data driven model that learns to perform a retrosynthetic
reaction prediction task, which is treated as a sequence-to-sequence mapping
problem. The end-to-end trained model has an encoder-decoder architecture that
consists of two recurrent neural networks, which has previously shown great
success in solving other sequence-to-sequence prediction tasks such as machine
translation. The model is trained on 50,000 experimental reaction examples from
the United States patent literature, which span 10 broad reaction types that
are commonly used by medicinal chemists. We find that our model performs
comparably with a rule-based expert system baseline model, and also overcomes
certain limitations associated with rule-based expert systems and with any
machine learning approach that contains a rule-based expert system component.
Our model provides an important first step towards solving the challenging
problem of computational retrosynthetic analysis.
Recommendation with k-anonymized Ratings
Authors: Jun Sakuma, Tatsuya Osame
Date: 2017-06-06 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Recommender systems are widely used to predict personalized preferences of
goods or services using users' past activities, such as item ratings or
purchase histories. If collections of such personal activities were made
publicly available, they could be used to personalize a diverse range of
services, including targeted advertisement or recommendations. However, there
would be an accompanying risk of privacy violations. The pioneering work of
Narayanan et al.\ demonstrated that even if the identifiers are eliminated, the
public release of user ratings can allow for the identification of users by
those who have only a small amount of data on the users' past ratings.
  In this paper, we assume the following setting. A collector collects user
ratings, then anonymizes and distributes them. A recommender constructs a
recommender system based on the anonymized ratings provided by the collector.
Based on this setting, we exhaustively list the models of recommender systems
that use anonymized ratings. For each model, we then present an item-based
collaborative filtering algorithm for making recommendations based on
anonymized ratings. Our experimental results show that an item-based
collaborative filtering based on anonymized ratings can perform better than
collaborative filterings based on 5--10 non-anonymized ratings. This surprising
result indicates that, in some settings, privacy protection does not
necessarily reduce the usefulness of recommendations. From the experimental
analysis of this counterintuitive result, we observed that the sparsity of the
ratings can be reduced by anonymization and the variance of the prediction can
be reduced if $k$, the anonymization parameter, is appropriately tuned. In this
way, the predictive performance of recommendations based on anonymized ratings
can be improved in some settings.
Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning
Authors: Jiasen Lu, Caiming Xiong, Devi Parikh, Richard Socher
Date: 2016-12-06 00:00:00
Proceeding: CVPR 2017 7
Tasks: Decoder, Image Captioning, Language Modeling, Language Modelling

Attention-based neural encoder-decoder frameworks have been widely adopted
for image captioning. Most methods force visual attention to be active for
every generated word. However, the decoder likely requires little to no visual
information from the image to predict non-visual words such as "the" and "of".
Other words that may seem visual can often be predicted reliably just from the
language model e.g., "sign" after "behind a red stop" or "phone" following
"talking on a cell". In this paper, we propose a novel adaptive attention model
with a visual sentinel. At each time step, our model decides whether to attend
to the image (and if so, to which regions) or to the visual sentinel. The model
decides whether to attend to the image and where, in order to extract
meaningful information for sequential word generation. We test our method on
the COCO image captioning 2015 challenge dataset and Flickr30K. Our approach
sets the new state-of-the-art by a significant margin.
DeepKey: An EEG and Gait Based Dual-Authentication System
Authors: Xiang Zhang, Lina Yao, Chaoran Huang, Tao Gu, Zheng Yang, Yunhao Liu
Date: 2017-06-06 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Face Recognition, Gait Identification

Biometric authentication involves various technologies to identify individuals by exploiting their unique, measurable physiological and behavioral characteristics. However, traditional biometric authentication systems (e.g., face recognition, iris, retina, voice, and fingerprint) are facing an increasing risk of being tricked by biometric tools such as anti-surveillance masks, contact lenses, vocoder, or fingerprint films. In this paper, we design a multimodal biometric authentication system named Deepkey, which uses both Electroencephalography (EEG) and gait signals to better protect against such risk. Deepkey consists of two key components: an Invalid ID Filter Model to block unauthorized subjects and an identification model based on attention-based Recurrent Neural Network (RNN) to identify a subject`s EEG IDs and gait IDs in parallel. The subject can only be granted access while all the components produce consistent evidence to match the user`s proclaimed identity. We implement Deepkey with a live deployment in our university and conduct extensive empirical experiments to study its technical feasibility in practice. DeepKey achieves the False Acceptance Rate (FAR) and the False Rejection Rate (FRR) of 0 and 1.0%, respectively. The preliminary results demonstrate that Deepkey is feasible, show consistent superior performance compared to a set of methods, and has the potential to be applied to the authentication deployment in real world settings.
Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory
Authors: Peter Richtárik, Martin Takáč
Date: 2017-06-04 00:00:00
Tasks: Stochastic Optimization

We develop a family of reformulations of an arbitrary consistent linear system into a stochastic problem. The reformulations are governed by two user-defined parameters: a positive definite matrix defining a norm, and an arbitrary discrete or continuous distribution over random matrices. Our reformulation has several equivalent interpretations, allowing for researchers from various communities to leverage their domain specific insights. In particular, our reformulation can be equivalently seen as a stochastic optimization problem, stochastic linear system, stochastic fixed point problem and a probabilistic intersection problem. We prove sufficient, and necessary and sufficient conditions for the reformulation to be exact. Further, we propose and analyze three stochastic algorithms for solving the reformulated problem---basic, parallel and accelerated methods---with global linear convergence rates. The rates can be interpreted as condition numbers of a matrix which depends on the system matrix and on the reformulation parameters. This gives rise to a new phenomenon which we call stochastic preconditioning, and which refers to the problem of finding parameters (matrix and distribution) leading to a sufficiently small condition number. Our basic method can be equivalently interpreted as stochastic gradient descent, stochastic Newton method, stochastic proximal point method, stochastic fixed point method, and stochastic projection method, with fixed stepsize (relaxation parameter), applied to the reformulations.