pwc-archive/papers-with-abstracts huggingface dataset:
Correlation Alignment for Unsupervised Domain Adaptation
Authors: Baochen Sun, Jiashi Feng, Kate Saenko
Date: 2016-12-06 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

In this chapter, we present CORrelation ALignment (CORAL), a simple yet
effective method for unsupervised domain adaptation. CORAL minimizes domain
shift by aligning the second-order statistics of source and target
distributions, without requiring any target labels. In contrast to subspace
manifold methods, it aligns the original feature distributions of the source
and target domains, rather than the bases of lower-dimensional subspaces. It is
also much simpler than other distribution matching methods. CORAL performs
remarkably well in extensive evaluations on standard benchmark datasets. We
first describe a solution that applies a linear transformation to source
features to align them with target features before classifier training. For
linear classifiers, we propose to equivalently apply CORAL to the classifier
weights, leading to added efficiency when the number of classifiers is small
but the number and dimensionality of target examples are very high. The
resulting CORAL Linear Discriminant Analysis (CORAL-LDA) outperforms LDA by a
large margin on standard domain adaptation benchmarks. Finally, we extend CORAL
to learn a nonlinear transformation that aligns correlations of layer
activations in deep neural networks (DNNs). The resulting Deep CORAL approach
works seamlessly with DNNs and achieves state-of-the-art performance on
standard benchmark datasets. Our code is available
at:~\url{https://github.com/VisionLearningGroup/CORAL}
A Probabilistic Framework for Deep Learning
Authors: Ankit B. Patel, Tan Nguyen, Richard G. Baraniuk
Date: 2016-12-06 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Deep Learning, General Classification

We develop a probabilistic framework for deep learning based on the Deep
Rendering Mixture Model (DRMM), a new generative probabilistic model that
explicitly capture variations in data due to latent task nuisance variables. We
demonstrate that max-sum inference in the DRMM yields an algorithm that exactly
reproduces the operations in deep convolutional neural networks (DCNs),
providing a first principles derivation. Our framework provides new insights
into the successes and shortcomings of DCNs as well as a principled route to
their improvement. DRMM training via the Expectation-Maximization (EM)
algorithm is a powerful alternative to DCN back-propagation, and initial
training results are promising. Classification based on the DRMM and other
variants outperforms DCNs in supervised digit classification, training 2-3x
faster while achieving similar accuracy. Moreover, the DRMM is applicable to
semi-supervised and unsupervised learning tasks, achieving results that are
state-of-the-art in several categories on the MNIST benchmark and comparable to
state of the art on the CIFAR10 benchmark.
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Authors: Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, Thomas Brox
Date: 2016-12-06 00:00:00
Proceeding: CVPR 2017 7
Tasks: Dense Pixel Correspondence Estimation, Optical Flow Estimation, Skeleton Based Action Recognition

The FlowNet demonstrated that optical flow estimation can be cast as a
learning problem. However, the state of the art with regard to the quality of
the flow has still been defined by traditional methods. Particularly on small
displacements and real-world data, FlowNet cannot compete with variational
methods. In this paper, we advance the concept of end-to-end learning of
optical flow and make it work really well. The large improvements in quality
and speed are caused by three major contributions: first, we focus on the
training data and show that the schedule of presenting data during training is
very important. Second, we develop a stacked architecture that includes warping
of the second image with intermediate optical flow. Third, we elaborate on
small displacements by introducing a sub-network specializing on small motions.
FlowNet 2.0 is only marginally slower than the original FlowNet but decreases
the estimation error by more than 50%. It performs on par with state-of-the-art
methods, while running at interactive frame rates. Moreover, we present faster
variants that allow optical flow computation at up to 140fps with accuracy
matching the original FlowNet.
Cross-Lingual Predicate Mapping Between Linked Data Ontologies
Authors: Gautam Singh, Saemi Jang, Mun Y. Yi
Date: 2016-12-06 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity

Ontologies in different natural languages often differ in quality in terms of
richness of schema or richness of internal links. This difference is markedly
visible when comparing a rich English language ontology with a non-English
language counterpart. Discovering alignment between them is a useful endeavor
as it serves as a starting point in bridging the disparity. In particular, our
work is motivated by the absence of inter-language links for predicates in the
localised versions of DBpedia. In this paper, we propose and demonstrate an
ad-hoc system to find possible owl:equivalentProperty links between predicates
in ontologies of different natural languages. We seek to achieve this mapping
by using pre-existing inter-language links of the resources connected by the
given predicate. Thus, our methodology stresses on semantic similarity rather
than lexical. Moreover, through an evaluation, we show that our system is
capable of outperforming a baseline system that is similar to the one used in
recent OAEI campaigns.
Recommender Engine for Continuous Time Quantum Monte Carlo Methods
Authors: Li Huang, Yi-feng Yang, Lei Wang
Date: 2016-12-06 00:00:00
Tasks: Recommendation Systems

Recommender systems play an essential role in the modern business world. They
recommend favorable items like books, movies, and search queries to users based
on their past preferences. Applying similar ideas and techniques to Monte Carlo
simulations of physical systems boosts their efficiency without sacrificing
accuracy. Exploiting the quantum to classical mapping inherent in the
continuous-time quantum Monte Carlo methods, we construct a classical molecular
gas model to reproduce the quantum distributions. We then utilize powerful
molecular simulation techniques to propose efficient quantum Monte Carlo
updates. The recommender engine approach provides a general way to speed up the
quantum impurity solvers.
Combinatorial semi-bandit with known covariance
Authors: Rémy Degenne, Vianney Perchet
Date: 2016-12-06 00:00:00
Proceeding: NeurIPS 2016 12

The combinatorial stochastic semi-bandit problem is an extension of the
classical multi-armed bandit problem in which an algorithm pulls more than one
arm at each stage and the rewards of all pulled arms are revealed. One
difference with the single arm variant is that the dependency structure of the
arms is crucial. Previous works on this setting either used a worst-case
approach or imposed independence of the arms. We introduce a way to quantify
the dependency structure of the problem and design an algorithm that adapts to
it. The algorithm is based on linear regression and the analysis develops
techniques from the linear bandit literature. By comparing its performance to a
new lower bound, we prove that it is optimal, up to a poly-logarithmic factor
in the number of pulled arms.
Sampling Generative Networks
Authors: Tom White
Date: 2016-09-14 00:00:00
Tasks: Attribute, Binary Classification, Data Augmentation, General Classification, Super-Resolution

We introduce several techniques for sampling and visualizing the latent
spaces of generative models. Replacing linear interpolation with spherical
linear interpolation prevents diverging from a model's prior distribution and
produces sharper samples. J-Diagrams and MINE grids are introduced as
visualizations of manifolds created by analogies and nearest neighbors. We
demonstrate two new techniques for deriving attribute vectors: bias-corrected
vectors with data replication and synthetic vectors with data augmentation.
Binary classification using attribute vectors is presented as a technique
supporting quantitative analysis of the latent space. Most techniques are
intended to be independent of model type and examples are shown on both
Variational Autoencoders and Generative Adversarial Networks.
Contextual RNN-GANs for Abstract Reasoning Diagram Generation
Authors: Arnab Ghosh, Viveka Kulharia, Amitabha Mukerjee, Vinay Namboodiri, Mohit Bansal
Date: 2016-09-29 00:00:00
Tasks: Generative Adversarial Network, Video Generation

Understanding, predicting, and generating object motions and transformations
is a core problem in artificial intelligence. Modeling sequences of evolving
images may provide better representations and models of motion and may
ultimately be used for forecasting, simulation, or video generation.
Diagrammatic Abstract Reasoning is an avenue in which diagrams evolve in
complex patterns and one needs to infer the underlying pattern sequence and
generate the next image in the sequence. For this, we develop a novel
Contextual Generative Adversarial Network based on Recurrent Neural Networks
(Context-RNN-GANs), where both the generator and the discriminator modules are
based on contextual history (modeled as RNNs) and the adversarial discriminator
guides the generator to produce realistic images for the particular time step
in the image sequence. We evaluate the Context-RNN-GAN model (and its variants)
on a novel dataset of Diagrammatic Abstract Reasoning, where it performs
competitively with 10th-grade human performance but there is still scope for
interesting improvements as compared to college-grade human performance. We
also evaluate our model on a standard video next-frame prediction task,
achieving improved performance over comparable state-of-the-art.
Analysis of the Human-Computer Interaction on the Example of Image-based CAPTCHA by Association Rule Mining
Authors: Darko Brodić, Alessia Amelio
Date: 2016-12-01 00:00:00

The paper analyzes the interaction between humans and computers in terms of
response time in solving the image-based CAPTCHA. In particular, the analysis
focuses on the attitude of the different Internet users in easily solving four
different types of image-based CAPTCHAs which include facial expressions like:
animated character, old woman, surprised face, worried face. To pursue this
goal, an experiment is realized involving 100 Internet users in solving the
four types of CAPTCHAs, differentiated by age, Internet experience, and
education level. The response times are collected for each user. Then,
association rules are extracted from user data, for evaluating the dependence
of the response time in solving the CAPTCHA from age, education level and
experience in internet usage by statistical analysis. The results implicitly
capture the users' psychological states showing in what states the users are
more sensible. It reveals to be a novelty and a meaningful analysis in the
state-of-the-art.
ImageNet pre-trained models with batch normalization
Authors: Marcel Simon, Erik Rodner, Joachim Denzler
Date: 2016-12-05 00:00:00

Convolutional neural networks (CNN) pre-trained on ImageNet are the backbone
of most state-of-the-art approaches. In this paper, we present a new set of
pre-trained models with popular state-of-the-art architectures for the Caffe
framework. The first release includes Residual Networks (ResNets) with
generation script as well as the batch-normalization-variants of AlexNet and
VGG19. All models outperform previous models with the same architecture. The
models and training code are available at
http://www.inf-cv.uni-jena.de/Research/CNN+Models.html and
https://github.com/cvjena/cnn-models
Localized Coulomb Descriptors for the Gaussian Approximation Potential
Authors: James Barker, Johannes Bulin, Jan Hamaekers, Sonja Mathias
Date: 2016-11-16 00:00:00

We introduce a novel class of localized atomic environment representations,
based upon the Coulomb matrix. By combining these functions with the Gaussian
approximation potential approach, we present LC-GAP, a new system for
generating atomic potentials through machine learning (ML). Tests on the QM7,
QM7b and GDB9 biomolecular datasets demonstrate that potentials created with
LC-GAP can successfully predict atomization energies for molecules larger than
those used for training to chemical accuracy, and can (in the case of QM7b)
also be used to predict a range of other atomic properties with accuracy in
line with the recent literature. As the best-performing representation has only
linear dimensionality in the number of atoms in a local atomic environment,
this represents an improvement both in prediction accuracy and computational
cost when considered against similar Coulomb matrix-based methods.
Factored Contextual Policy Search with Bayesian Optimization
Authors: Peter Karkus, Andras Kupcsik, David Hsu, Wee Sun Lee
Date: 2016-12-06 00:00:00
Tasks: Active Learning, Bayesian Optimization, Position, Vocal Bursts Type Prediction

Scarce data is a major challenge to scaling robot learning to truly complex tasks, as we need to generalize locally learned policies over different "contexts". Bayesian optimization approaches to contextual policy search (CPS) offer data-efficient policy learning that generalize over a context space. We propose to improve data-efficiency by factoring typically considered contexts into two components: target-type contexts that correspond to a desired outcome of the learned behavior, e.g. target position for throwing a ball; and environment type contexts that correspond to some state of the environment, e.g. initial ball position or wind speed. Our key observation is that experience can be directly generalized over target-type contexts. Based on that we introduce Factored Contextual Policy Search with Bayesian Optimization for both passive and active learning settings. Preliminary results show faster policy generalization on a simulated toy problem. A full paper extension is available at arXiv:1904.11761
Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation
Authors: Alexandre Berard, Olivier Pietquin, Christophe Servan, Laurent Besacier
Date: 2016-12-06 00:00:00
Tasks: Speech-to-Text, Speech-to-Text Translation, Translation

This paper proposes a first attempt to build an end-to-end speech-to-text
translation system, which does not use source language transcription during
learning or decoding. We propose a model for direct speech-to-text translation,
which gives promising results on a small French-English synthetic corpus.
Relaxing the need for source language transcription would drastically change
the data collection methodology in speech translation, especially in
under-resourced scenarios. For instance, in the former project DARPA TRANSTAC
(speech translation from spoken Arabic dialects), a large effort was devoted to
the collection of speech transcripts (and a prerequisite to obtain transcripts
was often a detailed transcription guide for languages with little standardized
spelling). Now, if end-to-end approaches for speech-to-text translation are
successful, one might consider collecting data by asking bilingual speakers to
directly utter speech in the source language from target language text
utterances. Such an approach has the advantage to be applicable to any
unwritten (source) language.
Generalizing the Kelly strategy
Authors: Arjun Viswanathan
Date: 2016-11-28 00:00:00

Prompted by a recent experiment by Victor Haghani and Richard Dewey, this
note generalises the Kelly strategy (optimal for simple investment games with
log utility) to a large class of practical utility functions and including the
effect of extraneous wealth.
  A counterintuitive result is proved : for any continuous, concave,
differentiable utility function, the optimal choice at every point depends only
on the probability of reaching that point.
  The practical calculation of the optimal action at every stage is made
possible through use of the binomial expansion, reducing the problem size from
exponential to quadratic.
  Applications include (better) automatic investing and risk taking under
uncertainty.
Review of state-of-the-arts in artificial intelligence with application to AI safety problem
Authors: Vladimir Shakirov
Date: 2016-05-11 00:00:00

Here, I review current state-of-the-arts in many areas of AI to estimate when
it's reasonable to expect human level AI development. Predictions of prominent
AI researchers vary broadly from very pessimistic predictions of Andrew Ng to
much more moderate predictions of Geoffrey Hinton and optimistic predictions of
Shane Legg, DeepMind cofounder. Given huge rate of progress in recent years and
this broad range of predictions of AI experts, AI safety questions are also
discussed.
Fleet Size and Mix Split-Delivery Vehicle Routing
Authors: Arthur Mahéo, Tommaso Urli, Philip Kilby
Date: 2016-12-06 00:00:00
Tasks: valid

In the classic Vehicle Routing Problem (VRP) a fleet of of vehicles has to
visit a set of customers while minimising the operations' costs. We study a
rich variant of the VRP featuring split deliveries, an heterogeneous fleet, and
vehicle-commodity incompatibility constraints. Our goal is twofold: define the
cheapest routing and the most adequate fleet.
  To do so, we split the problem into two interdependent components: a fleet
design component and a routing component. First, we define two Mixed Integer
Programming (MIP) formulations for each component. Then we discuss several
improvements in the form of valid cuts and symmetry breaking constraints.
  The main contribution of this paper is a comparison of the four resulting
models for this Rich VRP. We highlight their strengths and weaknesses with
extensive experiments.
  Finally, we explore a lightweight integration with Constraint Programming
(CP). We use a fast CP model which gives good solutions and use the solution to
warm-start our models.
Learning to Segment Object Candidates via Recursive Neural Networks
Authors: Tianshui Chen, Liang Lin, Xian Wu, Nong Xiao, Xiaonan Luo
Date: 2016-12-04 00:00:00
Tasks: Object, object-detection, Object Detection, Object Proposal Generation

To avoid the exhaustive search over locations and scales, current
state-of-the-art object detection systems usually involve a crucial component
generating a batch of candidate object proposals from images. In this paper, we
present a simple yet effective approach for segmenting object proposals via a
deep architecture of recursive neural networks (ReNNs), which hierarchically
groups regions for detecting object candidates over scales. Unlike traditional
methods that mainly adopt fixed similarity measures for merging regions or
finding object proposals, our approach adaptively learns the region merging
similarity and the objectness measure during the process of hierarchical region
grouping. Specifically, guided by a structured loss, the ReNN model jointly
optimizes the cross-region similarity metric with the region merging process as
well as the objectness prediction. During inference of the object proposal
generation, we introduce randomness into the greedy search to cope with the
ambiguity of grouping regions. Extensive experiments on standard benchmarks,
e.g., PASCAL VOC and ImageNet, suggest that our approach is capable of
producing object proposals with high recall while well preserving the object
boundaries and outperforms other existing methods in both accuracy and
efficiency.
Supervised topic models for clinical interpretability
Authors: Michael C. Hughes, Huseyin Melih Elibol, Thomas McCoy, Roy Perlis, Finale Doshi-Velez
Date: 2016-12-06 00:00:00
Tasks: Diagnostic, Topic Models

Supervised topic models can help clinical researchers find interpretable
cooccurence patterns in count data that are relevant for diagnostics. However,
standard formulations of supervised Latent Dirichlet Allocation have two
problems. First, when documents have many more words than labels, the influence
of the labels will be negligible. Second, due to conditional independence
assumptions in the graphical model the impact of supervised labels on the
learned topic-word probabilities is often minimal, leading to poor predictions
on heldout data. We investigate penalized optimization methods for training
sLDA that produce interpretable topic-word parameters and useful heldout
predictions, using recognition networks to speed-up inference. We report
preliminary results on synthetic data and on predicting successful
anti-depressant medication given a patient's diagnostic history.
Efficient Non-oblivious Randomized Reduction for Risk Minimization with Improved Excess Risk Guarantee
Authors: Yi Xu, Haiqin Yang, Lijun Zhang, Tianbao Yang
Date: 2016-12-06 00:00:00
Tasks: BIG-bench Machine Learning

In this paper, we address learning problems for high dimensional data.
Previously, oblivious random projection based approaches that project high
dimensional features onto a random subspace have been used in practice for
tackling high-dimensionality challenge in machine learning. Recently, various
non-oblivious randomized reduction methods have been developed and deployed for
solving many numerical problems such as matrix product approximation, low-rank
matrix approximation, etc. However, they are less explored for the machine
learning tasks, e.g., classification. More seriously, the theoretical analysis
of excess risk bounds for risk minimization, an important measure of
generalization performance, has not been established for non-oblivious
randomized reduction methods. It therefore remains an open problem what is the
benefit of using them over previous oblivious random projection based
approaches. To tackle these challenges, we propose an algorithmic framework for
employing non-oblivious randomized reduction method for general empirical risk
minimizing in machine learning tasks, where the original high-dimensional
features are projected onto a random subspace that is derived from the data
with a small matrix approximation error. We then derive the first excess risk
bound for the proposed non-oblivious randomized reduction approach without
requiring strong assumptions on the training data. The established excess risk
bound exhibits that the proposed approach provides much better generalization
performance and it also sheds more insights about different randomized
reduction approaches. Finally, we conduct extensive experiments on both
synthetic and real-world benchmark datasets, whose dimension scales to
$O(10^7)$, to demonstrate the efficacy of our proposed approach.
Awesome Typography: Statistics-Based Text Effects Transfer
Authors: Shuai Yang, Jiaying Liu, Zhouhui Lian, Zongming Guo
Date: 2016-11-28 00:00:00
Proceeding: CVPR 2017 7
Tasks: Style Transfer, Text Effects Transfer, Texture Synthesis

In this work, we explore the problem of generating fantastic special-effects
for the typography. It is quite challenging due to the model diversities to
illustrate varied text effects for different characters. To address this issue,
our key idea is to exploit the analytics on the high regularity of the spatial
distribution for text effects to guide the synthesis process. Specifically, we
characterize the stylized patches by their normalized positions and the optimal
scales to depict their style elements. Our method first estimates these two
features and derives their correlation statistically. They are then converted
into soft constraints for texture transfer to accomplish adaptive multi-scale
texture synthesis and to make style element distribution uniform. It allows our
algorithm to produce artistic typography that fits for both local texture
patterns and the global spatial distribution in the example. Experimental
results demonstrate the superiority of our method for various text effects over
conventional style transfer methods. In addition, we validate the effectiveness
of our algorithm with extensive artistic typography library generation.