pwc-archive/papers-with-abstracts huggingface dataset:
A Nuclear-norm Model for Multi-Frame Super-Resolution Reconstruction from Video Clips
Authors: Rui Zhao, Raymond H. Chan
Date: 2017-04-17 00:00:00
Tasks: Multi-Frame Super-Resolution, Optical Flow Estimation, Super-Resolution

We propose a variational approach to obtain super-resolution images from
multiple low-resolution frames extracted from video clips. First the
displacement between the low-resolution frames and the reference frame are
computed by an optical flow algorithm. Then a low-rank model is used to
construct the reference frame in high-resolution by incorporating the
information of the low-resolution frames. The model has two terms: a 2-norm
data fidelity term and a nuclear-norm regularization term. Alternating
direction method of multipliers is used to solve the model. Comparison of our
methods with other models on synthetic and real video clips show that our
resulting images are more accurate with less artifacts. It also provides much
finer and discernable details.
Fast Stochastic Variance Reduced Gradient Method with Momentum Acceleration for Machine Learning
Authors: Fanhua Shang, Yuanyuan Liu, James Cheng, Jiacheng Zhuo
Date: 2017-03-23 00:00:00
Tasks: BIG-bench Machine Learning, regression

Recently, research on accelerated stochastic gradient descent methods (e.g.,
SVRG) has made exciting progress (e.g., linear convergence for strongly convex
problems). However, the best-known methods (e.g., Katyusha) requires at least
two auxiliary variables and two momentum parameters. In this paper, we propose
a fast stochastic variance reduction gradient (FSVRG) method, in which we
design a novel update rule with the Nesterov's momentum and incorporate the
technique of growing epoch size. FSVRG has only one auxiliary variable and one
momentum weight, and thus it is much simpler and has much lower per-iteration
complexity. We prove that FSVRG achieves linear convergence for strongly convex
problems and the optimal $\mathcal{O}(1/T^2)$ convergence rate for non-strongly
convex problems, where $T$ is the number of outer-iterations. We also extend
FSVRG to directly solve the problems with non-smooth component functions, such
as SVM. Finally, we empirically study the performance of FSVRG for solving
various machine learning problems such as logistic regression, ridge
regression, Lasso and SVM. Our results show that FSVRG outperforms the
state-of-the-art stochastic methods, including Katyusha.
End-to-end 3D face reconstruction with deep neural networks
Authors: Pengfei Dou, Shishir K. Shah, Ioannis A. Kakadiaris
Date: 2017-04-17 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Face Reconstruction, Face Reconstruction

Monocular 3D facial shape reconstruction from a single 2D facial image has
been an active research area due to its wide applications. Inspired by the
success of deep neural networks (DNN), we propose a DNN-based approach for
End-to-End 3D FAce Reconstruction (UH-E2FAR) from a single 2D image. Different
from recent works that reconstruct and refine the 3D face in an iterative
manner using both an RGB image and an initial 3D facial shape rendering, our
DNN model is end-to-end, and thus the complicated 3D rendering process can be
avoided. Moreover, we integrate in the DNN architecture two components, namely
a multi-task loss function and a fusion convolutional neural network (CNN) to
improve facial expression reconstruction. With the multi-task loss function, 3D
face reconstruction is divided into neutral 3D facial shape reconstruction and
expressive 3D facial shape reconstruction. The neutral 3D facial shape is
class-specific. Therefore, higher layer features are useful. In comparison, the
expressive 3D facial shape favors lower or intermediate layer features. With
the fusion-CNN, features from different intermediate layers are fused and
transformed for predicting the 3D expressive facial shape. Through extensive
experiments, we demonstrate the superiority of our end-to-end framework in
improving the accuracy of 3D face reconstruction.
Morpheo: Traceable Machine Learning on Hidden data
Authors: Mathieu Galtier, Camille Marini
Date: 2017-04-17 00:00:00
Tasks: BIG-bench Machine Learning, Prediction, Transfer Learning

Morpheo is a transparent and secure machine learning platform collecting and
analysing large datasets. It aims at building state-of-the art prediction
models in various fields where data are sensitive. Indeed, it offers strong
privacy of data and algorithm, by preventing anyone to read the data, apart
from the owner and the chosen algorithms. Computations in Morpheo are
orchestrated by a blockchain infrastructure, thus offering total traceability
of operations. Morpheo aims at building an attractive economic ecosystem around
data prediction by channelling crypto-money from prediction requests to useful
data and algorithms providers. Morpheo is designed to handle multiple data
sources in a transfer learning approach in order to mutualize knowledge
acquired from large datasets for applications with smaller but similar
datasets.
(Yet) Another Theoretical Model of Thinking
Authors: Patrick Virie
Date: 2015-11-08 00:00:00
Tasks: model

This paper presents a theoretical, idealized model of the thinking process
with the following characteristics: 1) the model can produce complex thought
sequences and can be generalized to new inputs, 2) it can receive and maintain
input information indefinitely for the generation of thoughts and later use,
and 3) it supports learning while executing. The crux of the model lies within
the concept of internal consistency, or the generated thoughts should always be
consistent with the inputs from which they are created. Its merit, apart from
the capability to generate new creative thoughts from an internal mechanism,
depends on the potential to help training to generalize better. This is
consequently enabled by separating input information into several parts to be
handled by different processing components with a focus mechanism to fetch
information for each. This modularized view with the focus binds the model with
the computationally capable Turing machines. And as a final remark, this paper
constructively shows that the computational complexity of the model is at
least, if not surpass, that of a universal Turing machine.
Interval Arithmetic and Interval-Aware Operators for Genetic Programming
Authors: Grant Dick
Date: 2017-04-17 00:00:00
Tasks: regression, Symbolic Regression

Symbolic regression via genetic programming is a flexible approach to machine
learning that does not require up-front specification of model structure.
However, traditional approaches to symbolic regression require the use of
protected operators, which can lead to perverse model characteristics and poor
generalisation. In this paper, we revisit interval arithmetic as one possible
solution to allow genetic programming to perform regression using unprotected
operators. Using standard benchmarks, we show that using interval arithmetic
within model evaluation does not prevent invalid solutions from entering the
population, meaning that search performance remains compromised. We extend the
basic interval arithmetic concept with `safe' search operators that integrate
interval information into their process, thereby greatly reducing the number of
invalid solutions produced during search. The resulting algorithms are able to
more effectively identify good models that generalise well to unseen data. We
conclude with an analysis of the sensitivity of interval arithmetic-based
operators with respect to the accuracy of the supplied input feature intervals.
Multimodal Prediction and Personalization of Photo Edits with Deep Generative Models
Authors: Ardavan Saeedi, Matthew D. Hoffman, Stephen J. DiVerdi, Asma Ghandeharioun, Matthew J. Johnson, Ryan P. Adams
Date: 2017-04-17 00:00:00

Professional-grade software applications are powerful but
complicated$-$expert users can achieve impressive results, but novices often
struggle to complete even basic tasks. Photo editing is a prime example: after
loading a photo, the user is confronted with an array of cryptic sliders like
"clarity", "temp", and "highlights". An automatically generated suggestion
could help, but there is no single "correct" edit for a given image$-$different
experts may make very different aesthetic decisions when faced with the same
image, and a single expert may make different choices depending on the intended
use of the image (or on a whim). We therefore want a system that can propose
multiple diverse, high-quality edits while also learning from and adapting to a
user's aesthetic preferences. In this work, we develop a statistical model that
meets these objectives. Our model builds on recent advances in neural network
generative modeling and scalable inference, and uses hierarchical structure to
learn editing patterns across many diverse users. Empirically, we find that our
model outperforms other approaches on this challenging multimodal prediction
task.
Larger is Better: The Effect of Learning Rates Enjoyed by Stochastic Optimization with Progressive Variance Reduction
Authors: Fanhua Shang
Date: 2017-04-17 00:00:00
Tasks: Stochastic Optimization

In this paper, we propose a simple variant of the original stochastic
variance reduction gradient (SVRG), where hereafter we refer to as the variance
reduced stochastic gradient descent (VR-SGD). Different from the choices of the
snapshot point and starting point in SVRG and its proximal variant, Prox-SVRG,
the two vectors of each epoch in VR-SGD are set to the average and last iterate
of the previous epoch, respectively. This setting allows us to use much larger
learning rates or step sizes than SVRG, e.g., 3/(7L) for VR-SGD vs 1/(10L) for
SVRG, and also makes our convergence analysis more challenging. In fact, a
larger learning rate enjoyed by VR-SGD means that the variance of its
stochastic gradient estimator asymptotically approaches zero more rapidly.
Unlike common stochastic methods such as SVRG and proximal stochastic methods
such as Prox-SVRG, we design two different update rules for smooth and
non-smooth objective functions, respectively. In other words, VR-SGD can tackle
non-smooth and/or non-strongly convex problems directly without using any
reduction techniques such as quadratic regularizers. Moreover, we analyze the
convergence properties of VR-SGD for strongly convex problems, which show that
VR-SGD attains a linear convergence rate. We also provide the convergence
guarantees of VR-SGD for non-strongly convex problems. Experimental results
show that the performance of VR-SGD is significantly better than its
counterparts, SVRG and Prox-SVRG, and it is also much better than the best
known stochastic method, Katyusha.
Bayesian Hybrid Matrix Factorisation for Data Integration
Authors: Thomas Brouwer, Pietro Lió
Date: 2017-04-17 00:00:00
Tasks: Data Integration, Missing Values

We introduce a novel Bayesian hybrid matrix factorisation model (HMF) for
data integration, based on combining multiple matrix factorisation methods,
that can be used for in- and out-of-matrix prediction of missing values. The
model is very general and can be used to integrate many datasets across
different entity types, including repeated experiments, similarity matrices,
and very sparse datasets. We apply our method on two biological applications,
and extensively compare it to state-of-the-art machine learning and matrix
factorisation models. For in-matrix predictions on drug sensitivity datasets we
obtain consistently better performances than existing methods. This is
especially the case when we increase the sparsity of the datasets. Furthermore,
we perform out-of-matrix predictions on methylation and gene expression
datasets, and obtain the best results on two of the three datasets, especially
when the predictivity of datasets is high.
Adversarial and Clean Data Are Not Twins
Authors: Zhitao Gong, Wenlu Wang, Wei-Shinn Ku
Date: 2017-04-17 00:00:00
Tasks: Adversarial Attack

Adversarial attack has cast a shadow on the massive success of deep neural
networks. Despite being almost visually identical to the clean data, the
adversarial images can fool deep neural networks into wrong predictions with
very high confidence. In this paper, however, we show that we can build a
simple binary classifier separating the adversarial apart from the clean data
with accuracy over 99%. We also empirically show that the binary classifier is
robust to a second-round adversarial attack. In other words, it is difficult to
disguise adversarial samples to bypass the binary classifier. Further more, we
empirically investigate the generalization limitation which lingers on all
current defensive methods, including the binary classifier approach. And we
hypothesize that this is the result of intrinsic property of adversarial
crafting algorithms.
Introspection: Accelerating Neural Network Training By Learning Weight Evolution
Authors: Abhishek Sinha, Mausoom Sarkar, Aahitagni Mukherjee, Balaji Krishnamurthy
Date: 2017-04-17 00:00:00
Tasks: General Classification

Neural Networks are function approximators that have achieved
state-of-the-art accuracy in numerous machine learning tasks. In spite of their
great success in terms of accuracy, their large training time makes it
difficult to use them for various tasks. In this paper, we explore the idea of
learning weight evolution pattern from a simple network for accelerating
training of novel neural networks. We use a neural network to learn the
training pattern from MNIST classification and utilize it to accelerate
training of neural networks used for CIFAR-10 and ImageNet classification. Our
method has a low memory footprint and is computationally efficient. This method
can also be used with other optimizers to give faster convergence. The results
indicate a general trend in the weight evolution during training of neural
networks.
Error Asymmetry in Causal and Anticausal Regression
Authors: Patrick Blöbaum, Takashi Washio, Shohei Shimizu
Date: 2016-10-11 00:00:00
Tasks: Prediction, regression

It is generally difficult to make any statements about the expected
prediction error in an univariate setting without further knowledge about how
the data were generated. Recent work showed that knowledge about the real
underlying causal structure of a data generation process has implications for
various machine learning settings. Assuming an additive noise and an
independence between data generating mechanism and its input, we draw a novel
connection between the intrinsic causal relationship of two variables and the
expected prediction error. We formulate the theorem that the expected error of
the true data generating function as prediction model is generally smaller when
the effect is predicted from its cause and, on the contrary, greater when the
cause is predicted from its effect. The theorem implies an asymmetry in the
error depending on the prediction direction. This is further corroborated with
empirical evaluations in artificial and real-world data sets.
Exploration--Exploitation in MDPs with Options
Authors: Ronan Fruit, Alessandro Lazaric
Date: 2017-03-25 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

While a large body of empirical results show that temporally-extended actions
and options may significantly affect the learning performance of an agent, the
theoretical understanding of how and when options can be beneficial in online
reinforcement learning is relatively limited. In this paper, we derive an upper
and lower bound on the regret of a variant of UCRL using options. While we
first analyze the algorithm in the general case of semi-Markov decision
processes (SMDPs), we show how these results can be translated to the specific
case of MDPs with options and we illustrate simple scenarios in which the
regret of learning with options can be \textit{provably} much smaller than the
regret suffered when learning with primitive actions.
Pseudorehearsal in actor-critic agents
Authors: Marochko Vladimir, Leonard Johard, Manuel Mazzara
Date: 2017-04-17 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Catastrophic forgetting has a serious impact in reinforcement learning, as
the data distribution is generally sparse and non-stationary over time. The
purpose of this study is to investigate whether pseudorehearsal can increase
performance of an actor-critic agent with neural-network based policy selection
and function approximation in a pole balancing task and compare different
pseudorehearsal approaches. We expect that pseudorehearsal assists learning
even in such very simple problems, given proper initialization of the rehearsal
parameters.
Learning an Invariant Hilbert Space for Domain Adaptation
Authors: Samitha Herath, Mehrtash Harandi, Fatih Porikli
Date: 2016-11-25 00:00:00
Proceeding: CVPR 2017 7
Tasks: Domain Adaptation, Riemannian optimization, Semi-supervised Domain Adaptation

This paper introduces a learning scheme to construct a Hilbert space (i.e., a
vector space along its inner product) to address both unsupervised and
semi-supervised domain adaptation problems. This is achieved by learning
projections from each domain to a latent space along the Mahalanobis metric of
the latent space to simultaneously minimizing a notion of domain variance while
maximizing a measure of discriminatory power. In particular, we make use of the
Riemannian optimization techniques to match statistical properties (e.g., first
and second order statistics) between samples projected into the latent space
from different domains. Upon availability of class labels, we further deem
samples sharing the same label to form more compact clusters while pulling away
samples coming from different classes.We extensively evaluate and contrast our
proposal against state-of-the-art methods for the task of visual domain
adaptation using both handcrafted and deep-net features. Our experiments show
that even with a simple nearest neighbor classifier, the proposed method can
outperform several state-of-the-art methods benefitting from more involved
classification schemes.
Fostering User Engagement: Rhetorical Devices for Applause Generation Learnt from TED Talks
Authors: Zhe Liu, Anbang Xu, Mengdi Zhang, Jalal Mahmud, Vibha Sinha
Date: 2017-03-17 00:00:00
Tasks: regression

One problem that every presenter faces when delivering a public discourse is
how to hold the listeners' attentions or to keep them involved. Therefore, many
studies in conversation analysis work on this issue and suggest qualitatively
con-structions that can effectively lead to audience's applause. To investigate
these proposals quantitatively, in this study we an-alyze the transcripts of
2,135 TED Talks, with a particular fo-cus on the rhetorical devices that are
used by the presenters for applause elicitation. Through conducting regression
anal-ysis, we identify and interpret 24 rhetorical devices as triggers of
audience applauding. We further build models that can rec-ognize
applause-evoking sentences and conclude this work with potential implications.
Discriminative Bimodal Networks for Visual Localization and Detection with Natural Language Queries
Authors: Yuting Zhang, Luyao Yuan, Yijie Guo, Zhiyuan He, I-An Huang, Honglak Lee
Date: 2017-04-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: Natural Language Queries, Visual Localization

Associating image regions with text queries has been recently explored as a
new way to bridge visual and linguistic representations. A few pioneering
approaches have been proposed based on recurrent neural language models trained
generatively (e.g., generating captions), but achieving somewhat limited
localization accuracy. To better address natural-language-based visual entity
localization, we propose a discriminative approach. We formulate a
discriminative bimodal neural network (DBNet), which can be trained by a
classifier with extensive use of negative samples. Our training objective
encourages better localization on single images, incorporates text phrases in a
broad range, and properly pairs image regions with text phrases into positive
and negative examples. Experiments on the Visual Genome dataset demonstrate the
proposed DBNet significantly outperforms previous state-of-the-art methods both
for localization on single images and for detection on multiple images. We we
also establish an evaluation protocol for natural-language visual detection.
A General Characterization of the Statistical Query Complexity
Authors: Vitaly Feldman
Date: 2016-08-07 00:00:00
Tasks: Learning Theory

Statistical query (SQ) algorithms are algorithms that have access to an {\em
SQ oracle} for the input distribution $D$ instead of i.i.d.~ samples from $D$.
Given a query function $\phi:X \rightarrow [-1,1]$, the oracle returns an
estimate of ${\bf E}_{ x\sim D}[\phi(x)]$ within some tolerance $\tau_\phi$
that roughly corresponds to the number of samples.
  In this work we demonstrate that the complexity of solving general problems
over distributions using SQ algorithms can be captured by a relatively simple
notion of statistical dimension that we introduce. SQ algorithms capture a
broad spectrum of algorithmic approaches used in theory and practice, most
notably, convex optimization techniques. Hence our statistical dimension allows
to investigate the power of a variety of algorithmic approaches by analyzing a
single linear-algebraic parameter. Such characterizations were investigated
over the past 20 years in learning theory but prior characterizations are
restricted to the much simpler setting of classification problems relative to a
fixed distribution on the domain (Blum et al., 1994; Bshouty and Feldman, 2002;
Yang, 2001; Simon, 2007; Feldman, 2012; Szorenyi, 2009). Our characterization
is also the first to precisely characterize the necessary tolerance of queries.
We give applications of our techniques to two open problems in learning theory
and to algorithms that are subject to memory and communication constraints.
A Sport Tournament Scheduling by Genetic Algorithm with Swapping Method
Authors: Tinnaluk Rutjanisarakul, Thiradet Jiarasuksakun
Date: 2017-04-17 00:00:00
Tasks: Scheduling

A sport tournament problem is considered the Traveling Tournament Problem
(TTP). One interesting type is the mirrored Traveling Tournament Problem
(mTTP). The objective of the problem is to minimize either the total number of
traveling or the total distances of traveling or both. This research aims to
find an optimized solution of the mirrored Traveling Tournament Problem with
minimum total number of traveling. The solutions consisting of traveling and
scheduling tables are solved by using genetic algorithm (GA) with swapping
method. The number of traveling of all teams from obtained solutions are close
to the lower bound theory of number of traveling. Moreover, this algorithm
generates better solutions than known results for most cases.