pwc-archive/papers-with-abstracts huggingface dataset:
Deep transfer learning in the assessment of the quality of protein models
Authors: David Menéndez Hurtado, Karolis Uziela, Arne Elofsson
Date: 2018-04-17 00:00:00
Tasks: Transfer Learning

MOTIVATION: Proteins fold into complex structures that are crucial for their
biological functions. Experimental determination of protein structures is
costly and therefore limited to a small fraction of all known proteins. Hence,
different computational structure prediction methods are necessary for the
modelling of the vast majority of all proteins. In most structure prediction
pipelines, the last step is to select the best available model and to estimate
its accuracy. This model quality estimation problem has been growing in
importance during the last decade, and progress is believed to be important for
large scale modelling of proteins. The current generation of model quality
estimation programs performs well at separating incorrect and good models, but
fails to consistently identify the best possible model. State-of-the-art model
quality assessment methods use a combination of features that describe a model
and the agreement of the model with features predicted from the protein
sequence.
  RESULTS: We first introduce a deep neural network architecture to predict
model quality using significantly fewer input features than state-of-the-art
methods. Thereafter, we propose a methodology to train the deep network that
leverages the comparative structure of the problem. We also show the
possibility of applying transfer learning on databases of known protein
structures. We demonstrate its viability by reaching state-of-the-art
performance using only a reduced set of input features and a coarse description
of the models.
  AVAILABILITY: The code will be freely available for download at
github.com/ElofssonLab/ProQ4.
Seeing Beyond Appearance - Mapping Real Images into Geometrical Domains for Unsupervised CAD-based Recognition
Authors: Benjamin Planche, Sergey Zakharov, Ziyan Wu, Andreas Hutter, Harald Kosch, Slobodan Ilic
Date: 2018-10-09 00:00:00
Tasks: Denoising, Domain Adaptation, Object Recognition

While convolutional neural networks are dominating the field of computer
vision, one usually does not have access to the large amount of domain-relevant
data needed for their training. It thus became common to use available
synthetic samples along domain adaptation schemes to prepare algorithms for the
target domain. Tackling this problem from a different angle, we introduce a
pipeline to map unseen target samples into the synthetic domain used to train
task-specific methods. Denoising the data and retaining only the features these
recognition algorithms are familiar with, our solution greatly improves their
performance. As this mapping is easier to learn than the opposite one (ie to
learn to generate realistic features to augment the source samples), we
demonstrate how our whole solution can be trained purely on augmented synthetic
data, and still perform better than methods trained with domain-relevant
information (eg real images or realistic textures for the 3D models). Applying
our approach to object recognition from texture-less CAD data, we present a
custom generative network which fully utilizes the purely geometrical
information to learn robust features and achieve a more refined mapping for
unseen color images.
Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives
Authors: George Tucker, Dieterich Lawson, Shixiang Gu, Chris J. Maddison
Date: 2018-10-09 00:00:00
Proceeding: ICLR 2019 5
Tasks: Variational Inference

Deep latent variable models have become a popular model choice due to the
scalable learning algorithms introduced by (Kingma & Welling, 2013; Rezende et
al., 2014). These approaches maximize a variational lower bound on the
intractable log likelihood of the observed data. Burda et al. (2015) introduced
a multi-sample variational bound, IWAE, that is at least as tight as the
standard variational lower bound and becomes increasingly tight as the number
of samples increases. Counterintuitively, the typical inference network
gradient estimator for the IWAE bound performs poorly as the number of samples
increases (Rainforth et al., 2018; Le et al., 2018). Roeder et al. (2017)
propose an improved gradient estimator, however, are unable to show it is
unbiased. We show that it is in fact biased and that the bias can be estimated
efficiently with a second application of the reparameterization trick. The
doubly reparameterized gradient (DReG) estimator does not suffer as the number
of samples increases, resolving the previously raised issues. The same idea can
be used to improve many recently introduced training techniques for latent
variable models. In particular, we show that this estimator reduces the
variance of the IWAE gradient, the reweighted wake-sleep update (RWS)
(Bornschein & Bengio, 2014), and the jackknife variational inference (JVI)
gradient (Nowozin, 2018). Finally, we show that this computationally efficient,
unbiased drop-in gradient estimator translates to improved performance for all
three objectives on several modeling tasks.
Entropic GANs meet VAEs: A Statistical Approach to Compute Sample Likelihoods in GANs
Authors: Yogesh Balaji, Hamed Hassani, Rama Chellappa, Soheil Feizi
Date: 2018-10-09 00:00:00
Proceeding: ICLR 2019 5

Building on the success of deep learning, two modern approaches to learn a probability model from the data are Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs). VAEs consider an explicit probability model for the data and compute a generative distribution by maximizing a variational lower-bound on the log-likelihood function. GANs, however, compute a generative model by minimizing a distance between observed and generated probability distributions without considering an explicit model for the observed data. The lack of having explicit probability models in GANs prohibits computation of sample likelihoods in their frameworks and limits their use in statistical inference problems. In this work, we resolve this issue by constructing an explicit probability model that can be used to compute sample likelihood statistics in GANs. In particular, we prove that under this probability model, a family of Wasserstein GANs with an entropy regularization can be viewed as a generative model that maximizes a variational lower-bound on average sample log likelihoods, an approach that VAEs are based on. This result makes a principled connection between two modern generative models, namely GANs and VAEs. In addition to the aforementioned theoretical results, we compute likelihood statistics for GANs trained on Gaussian, MNIST, SVHN, CIFAR-10 and LSUN datasets. Our numerical results validate the proposed theory.
A Fast, Compact, Accurate Model for Language Identification of Codemixed Text
Authors: Yuan Zhang, Jason Riesa, Daniel Gillick, Anton Bakalov, Jason Baldridge, David Weiss
Date: 2018-10-09 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Decoder, Language Identification, Sentence

We address fine-grained multilingual language identification: providing a
language code for every token in a sentence, including codemixed text
containing multiple languages. Such text is prevalent online, in documents,
social media, and message boards. We show that a feed-forward network with a
simple globally constrained decoder can accurately and rapidly label both
codemixed and monolingual text in 100 languages and 100 language pairs. This
model outperforms previously published multilingual approaches in terms of both
accuracy and speed, yielding an 800x speed-up and a 19.5% averaged absolute
gain on three codemixed datasets. It furthermore outperforms several benchmark
systems on monolingual language identification.
Learning One-hidden-layer Neural Networks under General Input Distributions
Authors: Weihao Gao, Ashok Vardhan Makkuva, Sewoong Oh, Pramod Viswanath
Date: 2018-10-09 00:00:00

Significant advances have been made recently on training neural networks,
where the main challenge is in solving an optimization problem with abundant
critical points. However, existing approaches to address this issue crucially
rely on a restrictive assumption: the training data is drawn from a Gaussian
distribution. In this paper, we provide a novel unified framework to design
loss functions with desirable landscape properties for a wide range of general
input distributions. On these loss functions, remarkably, stochastic gradient
descent theoretically recovers the true parameters with global initializations
and empirically outperforms the existing approaches. Our loss function design
bridges the notion of score functions with the topic of neural network
optimization. Central to our approach is the task of estimating the score
function from samples, which is of basic and independent interest to
theoretical statistics. Traditional estimation methods (example: kernel based)
fail right at the outset; we bring statistical methods of local likelihood to
design a novel estimator of score functions, that provably adapts to the local
geometry of the unknown density.
Positional Cartesian Genetic Programming
Authors: DG Wilson, Julian F. Miller, Sylvain Cussat-Blanc, Hervé Luga
Date: 2018-10-09 00:00:00
Tasks: Form

Cartesian Genetic Programming (CGP) has many modifications across a variety
of implementations, such as recursive connections and node weights. Alternative
genetic operators have also been proposed for CGP, but have not been fully
studied. In this work, we present a new form of genetic programming based on a
floating point representation. In this new form of CGP, called Positional CGP,
node positions are evolved. This allows for the evaluation of many different
genetic operators while allowing for previous CGP improvements like recurrency.
Using nine benchmark problems from three different classes, we evaluate the
optimal parameters for CGP and PCGP, including novel genetic operators.
Semi-supervised Deep Reinforcement Learning in Support of IoT and Smart City Services
Authors: Mehdi Mohammadi, Ala Al-Fuqaha, Mohsen Guizani, Jun-Seok Oh
Date: 2018-10-09 00:00:00
Tasks: Deep Reinforcement Learning, Indoor Localization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Smart services are an important element of the smart cities and the Internet
of Things (IoT) ecosystems where the intelligence behind the services is
obtained and improved through the sensory data. Providing a large amount of
training data is not always feasible; therefore, we need to consider
alternative ways that incorporate unlabeled data as well. In recent years, Deep
reinforcement learning (DRL) has gained great success in several application
domains. It is an applicable method for IoT and smart city scenarios where
auto-generated data can be partially labeled by users' feedback for training
purposes. In this paper, we propose a semi-supervised deep reinforcement
learning model that fits smart city applications as it consumes both labeled
and unlabeled data to improve the performance and accuracy of the learning
agent. The model utilizes Variational Autoencoders (VAE) as the inference
engine for generalizing optimal policies. To the best of our knowledge, the
proposed model is the first investigation that extends deep reinforcement
learning to the semi-supervised paradigm. As a case study of smart city
applications, we focus on smart buildings and apply the proposed model to the
problem of indoor localization based on BLE signal strength. Indoor
localization is the main component of smart city services since people spend
significant time in indoor environments. Our model learns the best action
policies that lead to a close estimation of the target locations with an
improvement of 23% in terms of distance to the target and at least 67% more
received rewards compared to the supervised DRL model.
A Family of Maximum Margin Criterion for Adaptive Learning
Authors: Miao Cheng, Zunren Liu, Hongwei Zou, Ah Chung Tsoi
Date: 2018-10-09 00:00:00
Tasks: Diversity

In recent years, pattern analysis plays an important role in data mining and
recognition, and many variants have been proposed to handle complicated
scenarios. In the literature, it has been quite familiar with high
dimensionality of data samples, but either such characteristics or large data
have become usual sense in real-world applications. In this work, an improved
maximum margin criterion (MMC) method is introduced firstly. With the new
definition of MMC, several variants of MMC, including random MMC, layered MMC,
2D^2 MMC, are designed to make adaptive learning applicable. Particularly, the
MMC network is developed to learn deep features of images in light of simple
deep networks. Experimental results on a diversity of data sets demonstrate the
discriminant ability of proposed MMC methods are compenent to be adopted in
complicated application scenarios.
Discovering General-Purpose Active Learning Strategies
Authors: Ksenia Konyushkova, Raphael Sznitman, Pascal Fua
Date: 2018-10-09 00:00:00
Proceeding: ICLR 2019 5
Tasks: Active Learning, BIG-bench Machine Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a general-purpose approach to discovering active learning (AL)
strategies from data. These strategies are transferable from one domain to
another and can be used in conjunction with many machine learning models. To
this end, we formalize the annotation process as a Markov decision process,
design universal state and action spaces and introduce a new reward function
that precisely model the AL objective of minimizing the annotation cost. We
seek to find an optimal (non-myopic) AL strategy using reinforcement learning.
We evaluate the learned strategies on multiple unrelated domains and show that
they consistently outperform state-of-the-art baselines.
Real time expert system for anomaly detection of aerators based on computer vision technology and existing surveillance cameras
Authors: Yeqi Liu, Yingyi Chen, Huihui Yu, Xiaomin Fang, Chuanyang Gong
Date: 2018-10-09 00:00:00
Tasks: Anomaly Detection, Cultural Vocal Bursts Intensity Prediction, Dimensionality Reduction, Object, Region Proposal, Time Series Analysis

Aerators are essential and crucial auxiliary devices in intensive culture,
especially in industrial culture in China. The traditional methods cannot
accurately detect abnormal condition of aerators in time. Surveillance cameras
are widely used as visual perception modules of the Internet of Things, and
then using these widely existing surveillance cameras to realize real-time
anomaly detection of aerators is a cost-free and easy-to-promote method.
However, it is difficult to develop such an expert system due to some technical
and applied challenges, e.g., illumination, occlusion, complex background, etc.
To tackle these aforementioned challenges, we propose a real-time expert system
based on computer vision technology and existing surveillance cameras for
anomaly detection of aerators, which consists of two modules, i.e., object
region detection and working state detection. First, it is difficult to detect
the working state for some small object regions in whole images, and the time
complexity of global feature comparison is also high, so we present an object
region detection method based on the region proposal idea. Moreover, we propose
a novel algorithm called reference frame Kanade-Lucas-Tomasi (RF-KLT) algorithm
for motion feature extraction in fixed regions. Then, we present a dimension
reduction method of time series for establishing a feature dataset with obvious
boundaries between classes. Finally, we use machine learning algorithms to
build the feature classifier. The experimental results in both the actual video
dataset and the augmented video dataset show that the accuracy for detecting
object region and working state of aerators is 100% and 99.9% respectively, and
the detection speed is 77-333 frames per second (FPS) according to the
different types of surveillance cameras.
Enabling Cognitive Smart Cities Using Big Data and Machine Learning: Approaches and Challenges
Authors: Mehdi Mohammadi, Ala Al-Fuqaha
Date: 2018-10-09 00:00:00
Tasks: BIG-bench Machine Learning, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The development of smart cities and their fast-paced deployment is resulting
in the generation of large quantities of data at unprecedented rates.
Unfortunately, most of the generated data is wasted without extracting
potentially useful information and knowledge because of the lack of established
mechanisms and standards that benefit from the availability of such data.
Moreover, the high dynamical nature of smart cities calls for new generation of
machine learning approaches that are flexible and adaptable to cope with the
dynamicity of data to perform analytics and learn from real-time data. In this
article, we shed the light on the challenge of under utilizing the big data
generated by smart cities from a machine learning perspective. Especially, we
present the phenomenon of wasting unlabeled data. We argue that
semi-supervision is a must for smart city to address this challenge. We also
propose a three-level learning framework for smart cities that matches the
hierarchical nature of big data generated by smart cities with a goal of
providing different levels of knowledge abstractions. The proposed framework is
scalable to meet the needs of smart city services. Fundamentally, the framework
benefits from semi-supervised deep reinforcement learning where a small amount
of data that has users' feedback serves as labeled data while a larger amount
is without such users' feedback serves as unlabeled data. This paper also
explores how deep reinforcement learning and its shift toward semi-supervision
can handle the cognitive side of smart city services and improve their
performance by providing several use cases spanning the different domains of
smart cities. We also highlight several challenges as well as promising future
research directions for incorporating machine learning and high-level
intelligence into smart city services.
Image Captioning as Neural Machine Translation Task in SOCKEYE
Authors: Loris Bazzani, Tobias Domhan, Felix Hieber
Date: 2018-10-09 00:00:00
Tasks: Decoder, Image Captioning, Machine Translation, Sentence, Translation

Image captioning is an interdisciplinary research problem that stands between
computer vision and natural language processing. The task is to generate a
textual description of the content of an image. The typical model used for
image captioning is an encoder-decoder deep network, where the encoder captures
the essence of an image while the decoder is responsible for generating a
sentence describing the image. Attention mechanisms can be used to
automatically focus the decoder on parts of the image which are relevant to
predict the next word. In this paper, we explore different decoders and
attentional models popular in neural machine translation, namely attentional
recurrent neural networks, self-attentional transformers, and
fully-convolutional networks, which represent the current state of the art of
neural machine translation. The image captioning module is available as part of
SOCKEYE at https://github.com/awslabs/sockeye which tutorial can be found at
https://awslabs.github.io/sockeye/image_captioning.html .
Characterization of Convex Objective Functions and Optimal Expected Convergence Rates for SGD
Authors: Marten van Dijk, Lam M. Nguyen, Phuong Ha Nguyen, Dzung T. Phan
Date: 2018-10-09 00:00:00

We study Stochastic Gradient Descent (SGD) with diminishing step sizes for convex objective functions. We introduce a definitional framework and theory that defines and characterizes a core property, called curvature, of convex objective functions. In terms of curvature we can derive a new inequality that can be used to compute an optimal sequence of diminishing step sizes by solving a differential equation. Our exact solutions confirm known results in literature and allows us to fully characterize a new regularizer with its corresponding expected convergence rates.
Geometry meets semantics for semi-supervised monocular depth estimation
Authors: Pierluigi Zama Ramirez, Matteo Poggi, Fabio Tosi, Stefano Mattoccia, Luigi Di Stefano
Date: 2018-10-09 00:00:00
Tasks: Decoder, Depth Estimation, Depth Prediction, Monocular Depth Estimation, Semantic Segmentation

Depth estimation from a single image represents a very exciting challenge in
computer vision. While other image-based depth sensing techniques leverage on
the geometry between different viewpoints (e.g., stereo or structure from
motion), the lack of these cues within a single image renders ill-posed the
monocular depth estimation task. For inference, state-of-the-art
encoder-decoder architectures for monocular depth estimation rely on effective
feature representations learned at training time. For unsupervised training of
these models, geometry has been effectively exploited by suitable images
warping losses computed from views acquired by a stereo rig or a moving camera.
In this paper, we make a further step forward showing that learning semantic
information from images enables to improve effectively monocular depth
estimation as well. In particular, by leveraging on semantically labeled images
together with unsupervised signals gained by geometry through an image warping
loss, we propose a deep learning approach aimed at joint semantic segmentation
and depth estimation. Our overall learning framework is semi-supervised, as we
deploy groundtruth data only in the semantic domain. At training time, our
network learns a common feature representation for both tasks and a novel
cross-task loss function is proposed. The experimental findings show how,
jointly tackling depth prediction and semantic segmentation, allows to improve
depth estimation accuracy. In particular, on the KITTI dataset our network
outperforms state-of-the-art methods for monocular depth estimation.
Bridging the gap between regret minimization and best arm identification, with application to A/B tests
Authors: Rémy Degenne, Thomas Nedelec, Clément Calauzènes, Vianney Perchet
Date: 2018-10-09 00:00:00

State of the art online learning procedures focus either on selecting the
best alternative ("best arm identification") or on minimizing the cost (the
"regret"). We merge these two objectives by providing the theoretical analysis
of cost minimizing algorithms that are also delta-PAC (with a proven guaranteed
bound on the decision time), hence fulfilling at the same time regret
minimization and best arm identification. This analysis sheds light on the
common observation that ill-callibrated UCB-algorithms minimize regret while
still identifying quickly the best arm.
  We also extend these results to the non-iid case faced by many practitioners.
This provides a technique to make cost versus decision time compromise when
doing adaptive tests with applications ranging from website A/B testing to
clinical trials.
Deep learning with differential Gaussian process flows
Authors: Pashupati Hegde, Markus Heinonen, Harri Lähdesmäki, Samuel Kaski
Date: 2018-10-09 00:00:00
Tasks: Deep Learning, Gaussian Processes, General Classification, regression

We propose a novel deep learning paradigm of differential flows that learn a
stochastic differential equation transformations of inputs prior to a standard
classification or regression function. The key property of differential
Gaussian processes is the warping of inputs through infinitely deep, but
infinitesimal, differential fields, that generalise discrete layers into a
dynamical system. We demonstrate state-of-the-art results that exceed the
performance of deep Gaussian processes and neural networks