pwc-archive/papers-with-abstracts huggingface dataset:
Adaptive Regularization in Convex Composite Optimization for Variational Imaging Problems
Authors: Byung-Woo Hong, Ja-Keoung Koo, Hendrik Dirks, Martin Burger
Date: 2016-09-08 00:00:00
Tasks: Denoising, Motion Estimation

We propose an adaptive regularization scheme in a variational framework where
a convex composite energy functional is optimized. We consider a number of
imaging problems including denoising, segmentation and motion estimation, which
are considered as optimal solutions of the energy functionals that mainly
consist of data fidelity, regularization and a control parameter for their
trade-off. We presents an algorithm to determine the relative weight between
data fidelity and regularization based on the residual that measures how well
the observation fits the model. Our adaptive regularization scheme is designed
to locally control the regularization at each pixel based on the assumption
that the diversity of the residual of a given imaging model spatially varies.
The energy optimization is presented in the alternating direction method of
multipliers (ADMM) framework where the adaptive regularization is iteratively
applied along with mathematical analysis of the proposed algorithm. We
demonstrate the robustness and effectiveness of our adaptive regularization
through experimental results presenting that the qualitative and quantitative
evaluation results of each imaging task are superior to the results with a
constant regularization scheme. The desired properties, robustness and
effectiveness, of the regularization parameter selection in a variational
framework for imaging problems are achieved by merely replacing the static
regularization parameter with our adaptive one.
Evaluation of Machine Learning Methods to Predict Coronary Artery Disease Using Metabolomic Data
Authors: Henrietta Forssen, Riyaz S. Patel, Natalie Fitzpatrick, Aroon Hingorani, Adam Timmis, Harry Hemingway, Spiros C. Denaxas
Date: 2017-02-28 00:00:00
Tasks: BIG-bench Machine Learning, Disease Prediction, Prediction, regression

Metabolomic data can potentially enable accurate, non-invasive and low-cost
prediction of coronary artery disease. Regression-based analytical approaches
however might fail to fully account for interactions between metabolites, rely
on a priori selected input features and thus might suffer from poorer accuracy.
Supervised machine learning methods can potentially be used in order to fully
exploit the dimensionality and richness of the data. In this paper, we
systematically implement and evaluate a set of supervised learning methods (L1
regression, random forest classifier) and compare them to traditional
regression-based approaches for disease prediction using metabolomic data.
Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach
Authors: Satoshi Hara, Kohei Hayashi
Date: 2016-06-29 00:00:00
Tasks: Model Selection

Tree ensembles, such as random forests and boosted trees, are renowned for
their high prediction performance. However, their interpretability is
critically limited due to the enormous complexity. In this study, we present a
method to make a complex tree ensemble interpretable by simplifying the model.
Specifically, we formalize the simplification of tree ensembles as a model
selection problem. Given a complex tree ensemble, we aim at obtaining the
simplest representation that is essentially equivalent to the original one. To
this end, we derive a Bayesian model selection algorithm that optimizes the
simplified model while maintaining the prediction performance. Our numerical
experiments on several datasets showed that complicated tree ensembles were
reasonably approximated as interpretable.
Do Reichenbachian Common Cause Systems of Arbitrary Finite Size Exist?
Authors: Claudio Mazzola, Peter Evans
Date: 2017-02-28 00:00:00

The principle of common cause asserts that positive correlations between
causally unrelated events ought to be explained through the action of some
shared causal factors. Reichenbachian common cause systems are probabilistic
structures aimed at accounting for cases where correlations of the aforesaid
sort cannot be explained through the action of a single common cause. The
existence of Reichenbachian common cause systems of arbitrary finite size for
each pair of non-causally correlated events was allegedly demonstrated by
Hofer-Szab\'o and R\'edei in 2006. This paper shows that their proof is
logically deficient, and we propose an improved proof.
Optimal Categorical Attribute Transformation for Granularity Change in Relational Databases for Binary Decision Problems in Educational Data Mining
Authors: Paulo J. L. Adeodato, Fábio C. Pereira, Rosalvo F. Oliveira Neto
Date: 2017-02-28 00:00:00
Tasks: Attribute, regression

This paper presents an approach for transforming data granularity in
hierarchical databases for binary decision problems by applying regression to
categorical attributes at the lower grain levels. Attributes from a lower
hierarchy entity in the relational database have their information content
optimized through regression on the categories histogram trained on a small
exclusive labelled sample, instead of the usual mode category of the
distribution. The paper validates the approach on a binary decision task for
assessing the quality of secondary schools focusing on how logistic regression
transforms the students and teachers attributes into school attributes.
Experiments were carried out on Brazilian schools public datasets via 10-fold
cross-validation comparison of the ranking score produced also by logistic
regression. The proposed approach achieved higher performance than the usual
distribution mode transformation and equal to the expert weighing approach
measured by the maximum Kolmogorov-Smirnov distance and the area under the ROC
curve at 0.01 significance level.
Weakly- and Semi-Supervised Object Detection with Expectation-Maximization Algorithm
Authors: Ziang Yan, Jian Liang, Weishen Pan, Jin Li, Chang-Shui Zhang
Date: 2017-02-28 00:00:00
Tasks: object-detection, Object Detection, Semi-Supervised Object Detection

Object detection when provided image-level labels instead of instance-level
labels (i.e., bounding boxes) during training is an important problem in
computer vision, since large scale image datasets with instance-level labels
are extremely costly to obtain. In this paper, we address this challenging
problem by developing an Expectation-Maximization (EM) based object detection
method using deep convolutional neural networks (CNNs). Our method is
applicable to both the weakly-supervised and semi-supervised settings.
Extensive experiments on PASCAL VOC 2007 benchmark show that (1) in the weakly
supervised setting, our method provides significant detection performance
improvement over current state-of-the-art methods, (2) having access to a small
number of strongly (instance-level) annotated images, our method can almost
match the performace of the fully supervised Fast RCNN. We share our source
code at https://github.com/ZiangYan/EM-WSD.
Billion-scale similarity search with GPUs
Authors: Jeff Johnson, Matthijs Douze, Hervé Jégou
Date: 2017-02-28 00:00:00
Tasks: GPU, Image Similarity Search, Quantization

Similarity search finds application in specialized database systems handling
complex data such as images or videos, which are typically represented by
high-dimensional features and require specific indexing structures. This paper
tackles the problem of better utilizing GPUs for this task. While GPUs excel at
data-parallel tasks, prior approaches are bottlenecked by algorithms that
expose less parallelism, such as k-min selection, or make poor use of the
memory hierarchy.
  We propose a design for k-selection that operates at up to 55% of theoretical
peak performance, enabling a nearest neighbor implementation that is 8.5x
faster than prior GPU state of the art. We apply it in different similarity
search scenarios, by proposing optimized design for brute-force, approximate
and compressed-domain search based on product quantization. In all these
setups, we outperform the state of the art by large margins. Our implementation
enables the construction of a high accuracy k-NN graph on 95 million images
from the Yfcc100M dataset in 35 minutes, and of a graph connecting 1 billion
vectors in less than 12 hours on 4 Maxwell Titan X GPUs. We have open-sourced
our approach for the sake of comparison and reproducibility.
Improving the Neural GPU Architecture for Algorithm Learning
Authors: Karlis Freivalds, Renars Liepins
Date: 2017-02-28 00:00:00
Tasks: GPU

Algorithm learning is a core problem in artificial intelligence with
significant implications on automation level that can be achieved by machines.
Recently deep learning methods are emerging for synthesizing an algorithm from
its input-output examples, the most successful being the Neural GPU, capable of
learning multiplication. We present several improvements to the Neural GPU that
substantially reduces training time and improves generalization. We introduce a
new technique - hard nonlinearities with saturation costs- that has general
applicability. We also introduce a technique of diagonal gates that can be
applied to active-memory models. The proposed architecture is the first capable
of learning decimal multiplication end-to-end.
Stacked Thompson Bandits
Authors: Lenz Belzner, Thomas Gabor
Date: 2017-02-28 00:00:00
Tasks: Thompson Sampling

We introduce Stacked Thompson Bandits (STB) for efficiently generating plans
that are likely to satisfy a given bounded temporal logic requirement. STB uses
a simulation for evaluation of plans, and takes a Bayesian approach to using
the resulting information to guide its search. In particular, we show that
stacking multiarmed bandits and using Thompson sampling to guide the action
selection process for each bandit enables STB to generate plans that satisfy
requirements with a high probability while only searching a fraction of the
search space.
Bayesian Verification under Model Uncertainty
Authors: Lenz Belzner, Thomas Gabor
Date: 2017-02-28 00:00:00
Tasks: model

Machine learning enables systems to build and update domain models based on
runtime observations. In this paper, we study statistical model checking and
runtime verification for systems with this ability. Two challenges arise: (1)
Models built from limited runtime data yield uncertainty to be dealt with. (2)
There is no definition of satisfaction w.r.t. uncertain hypotheses. We propose
such a definition of subjective satisfaction based on recently introduced
satisfaction functions. We also propose the BV algorithm as a Bayesian solution
to runtime verification of subjective satisfaction under model uncertainty. BV
provides user-definable stochastic bounds for type I and II errors. We discuss
empirical results from an example application to illustrate our ideas.
Learning Deep Visual Object Models From Noisy Web Data: How to Make it Work
Authors: Nizar Massouh, Francesca Babiloni, Tatiana Tommasi, Jay Young, Nick Hawes, Barbara Caputo
Date: 2017-02-28 00:00:00
Proceeding: IEEE Xplore: 2017 12
Tasks: Object, Object Categorization, Object Discovery

Deep networks thrive when trained on large scale data collections. This has
given ImageNet a central role in the development of deep architectures for
visual object classification. However, ImageNet was created during a specific
period in time, and as such it is prone to aging, as well as dataset bias
issues. Moving beyond fixed training datasets will lead to more robust visual
systems, especially when deployed on robots in new environments which must
train on the objects they encounter there. To make this possible, it is
important to break free from the need for manual annotators. Recent work has
begun to investigate how to use the massive amount of images available on the
Web in place of manual image annotations. We contribute to this research thread
with two findings: (1) a study correlating a given level of noisily labels to
the expected drop in accuracy, for two deep architectures, on two different
types of noise, that clearly identifies GoogLeNet as a suitable architecture
for learning from Web data; (2) a recipe for the creation of Web datasets with
minimal noise and maximum visual variability, based on a visual and natural
language processing concept expansion strategy. By combining these two results,
we obtain a method for learning powerful deep object models automatically from
the Web. We confirm the effectiveness of our approach through object
categorization experiments using our Web-derived version of ImageNet on a
popular robot vision benchmark database, and on a lifelong object discovery
task on a mobile robot.
An Extensive Technique to Detect and Analyze Melanoma: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2017
Authors: G. Wiselin Jiji, P Johnson Durai Raj
Date: 2017-02-28 00:00:00
Tasks: General Classification, Segmentation

An automated method to detect and analyze the melanoma is presented to
improve diagnosis which will leads to the exact treatment. Image processing
techniques such as segmentation, feature descriptors and classification models
are involved in this method. In the First phase the lesion region is segmented
using CIELAB Color space Based Segmentation. Then feature descriptors such as
shape, color and texture are extracted. Finally, in the third phase lesion
region is classified as melanoma, seborrheic keratosis or nevus using multi
class O-A SVM model. Experiment with ISIC 2017 Archive skin image database has
been done and analyzed the results.
Generative Knowledge Transfer for Neural Language Models
Authors: Sungho Shin, Kyuyeon Hwang, Wonyong Sung
Date: 2016-08-14 00:00:00
Tasks: Language Modeling, Language Modelling, Text Generation, Transfer Learning

In this paper, we propose a generative knowledge transfer technique that
trains an RNN based language model (student network) using text and output
probabilities generated from a previously trained RNN (teacher network). The
text generation can be conducted by either the teacher or the student network.
We can also improve the performance by taking the ensemble of soft labels
obtained from multiple teacher networks. This method can be used for privacy
conscious language model adaptation because no user data is directly used for
training. Especially, when the soft labels of multiple devices are aggregated
via a trusted third party, we can expect very strong privacy protection.
MIML-FCN+: Multi-instance Multi-label Learning via Fully Convolutional Networks with Privileged Information
Authors: Hao Yang, Joey Tianyi Zhou, Jianfei Cai, Yew Soon Ong
Date: 2017-02-28 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Captioning, Multi-Label Learning, Object Recognition

Multi-instance multi-label (MIML) learning has many interesting applications
in computer visions, including multi-object recognition and automatic image
tagging. In these applications, additional information such as bounding-boxes,
image captions and descriptions is often available during training phrase,
which is referred as privileged information (PI). However, as existing works on
learning using PI only consider instance-level PI (privileged instances), they
fail to make use of bag-level PI (privileged bags) available in MIML learning.
Therefore, in this paper, we propose a two-stream fully convolutional network,
named MIML-FCN+, unified by a novel PI loss to solve the problem of MIML
learning with privileged bags. Compared to the previous works on PI, the
proposed MIML-FCN+ utilizes the readily available privileged bags, instead of
hard-to-obtain privileged instances, making the system more general and
practical in real world applications. As the proposed PI loss is convex and SGD
compatible and the framework itself is a fully convolutional network, MIML-FCN+
can be easily integrated with state of-the-art deep learning networks.
Moreover, the flexibility of convolutional layers allows us to exploit
structured correlations among instances to facilitate more effective training
and testing. Experimental results on three benchmark datasets demonstrate the
effectiveness of the proposed MIML-FCN+, outperforming state-of-the-art methods
in the application of multi-object recognition.
On architectural choices in deep learning: From network structure to gradient convergence and parameter estimation
Authors: Vamsi K. Ithapu, Sathya N. Ravi, Vikas Singh
Date: 2017-02-28 00:00:00
Tasks: Denoising, parameter estimation

We study mechanisms to characterize how the asymptotic convergence of
backpropagation in deep architectures, in general, is related to the network
structure, and how it may be influenced by other design choices including
activation type, denoising and dropout rate. We seek to analyze whether network
architecture and input data statistics may guide the choices of learning
parameters and vice versa. Given the broad applicability of deep architectures,
this issue is interesting both from theoretical and a practical standpoint.
Using properties of general nonconvex objectives (with first-order
information), we first build the association between structural, distributional
and learnability aspects of the network vis-\`a-vis their interaction with
parameter convergence rates. We identify a nice relationship between feature
denoising and dropout, and construct families of networks that achieve the same
level of convergence. We then derive a workflow that provides systematic
guidance regarding the choice of network sizes and learning parameters often
mediated4 by input statistics. Our technical results are corroborated by an
extensive set of evaluations, presented in this paper as well as independent
empirical observations reported by other groups. We also perform experiments
showing the practical implications of our framework for choosing the best
fully-connected design for a given problem.
Towards Deeper Understanding of Variational Autoencoding Models
Authors: Shengjia Zhao, Jiaming Song, Stefano Ermon
Date: 2017-02-28 00:00:00

We propose a new family of optimization criteria for variational
auto-encoding models, generalizing the standard evidence lower bound. We
provide conditions under which they recover the data distribution and learn
latent features, and formally show that common issues such as blurry samples
and uninformative latent features arise when these conditions are not met.
Based on these new insights, we propose a new sequential VAE model that can
generate sharp samples on the LSUN image dataset based on pixel-wise
reconstruction loss, and propose an optimization criterion that encourages
unsupervised learning of informative latent features.
Speeding Up Latent Variable Gaussian Graphical Model Estimation via Nonconvex Optimizations
Authors: Pan Xu, Jian Ma, Quanquan Gu
Date: 2017-02-28 00:00:00
Proceeding: NeurIPS 2017

We study the estimation of the latent variable Gaussian graphical model
(LVGGM), where the precision matrix is the superposition of a sparse matrix and
a low-rank matrix. In order to speed up the estimation of the sparse plus
low-rank components, we propose a sparsity constrained maximum likelihood
estimator based on matrix factorization, and an efficient alternating gradient
descent algorithm with hard thresholding to solve it. Our algorithm is orders
of magnitude faster than the convex relaxation based methods for LVGGM. In
addition, we prove that our algorithm is guaranteed to linearly converge to the
unknown sparse and low-rank components up to the optimal statistical precision.
Experiments on both synthetic and genomic data demonstrate the superiority of
our algorithm over the state-of-the-art algorithms and corroborate our theory.
Deep Coevolutionary Network: Embedding User and Item Features for Recommendation
Authors: Hanjun Dai, Yichen Wang, Rakshit Trivedi, Le Song
Date: 2016-09-13 00:00:00
Tasks: Activity Prediction, Network Embedding, Point Processes, Recommendation Systems

Recommender systems often use latent features to explain the behaviors of
users and capture the properties of items. As users interact with different
items over time, user and item features can influence each other, evolve and
co-evolve over time. The compatibility of user and item's feature further
influence the future interaction between users and items. Recently, point
process based models have been proposed in the literature aiming to capture the
temporally evolving nature of these latent features. However, these models
often make strong parametric assumptions about the evolution process of the
user and item latent features, which may not reflect the reality, and has
limited power in expressing the complex and nonlinear dynamics underlying these
processes. To address these limitations, we propose a novel deep coevolutionary
network model (DeepCoevolve), for learning user and item features based on
their interaction graph. DeepCoevolve use recurrent neural network (RNN) over
evolving networks to define the intensity function in point processes, which
allows the model to capture complex mutual influence between users and items,
and the feature evolution over time. We also develop an efficient procedure for
training the model parameters, and show that the learned models lead to
significant improvements in recommendation and activity prediction compared to
previous state-of-the-arts parametric models.
Learning What Data to Learn
Authors: Yang Fan, Fei Tian, Tao Qin, Jiang Bian, Tie-Yan Liu
Date: 2017-02-28 00:00:00
Tasks: BIG-bench Machine Learning, Deep Reinforcement Learning, image-classification, Image Classification, Reinforcement Learning

Machine learning is essentially the sciences of playing with data. An
adaptive data selection strategy, enabling to dynamically choose different data
at various training stages, can reach a more effective model in a more
efficient way. In this paper, we propose a deep reinforcement learning
framework, which we call \emph{\textbf{N}eural \textbf{D}ata \textbf{F}ilter}
(\textbf{NDF}), to explore automatic and adaptive data selection in the
training process. In particular, NDF takes advantage of a deep neural network
to adaptively select and filter important data instances from a sequential
stream of training data, such that the future accumulative reward (e.g., the
convergence speed) is maximized. In contrast to previous studies in data
selection that is mainly based on heuristic strategies, NDF is quite generic
and thus can be widely suitable for many machine learning tasks. Taking neural
network training with stochastic gradient descent (SGD) as an example,
comprehensive experiments with respect to various neural network modeling
(e.g., multi-layer perceptron networks, convolutional neural networks and
recurrent neural networks) and several applications (e.g., image classification
and text understanding) demonstrate that NDF powered SGD can achieve comparable
accuracy with standard SGD process by using less data and fewer iterations.
Show, Attend and Interact: Perceivable Human-Robot Social Interaction through Neural Attention Q-Network
Authors: Ahmed Hussain Qureshi, Yutaka Nakamura, Yuichiro Yoshikawa, Hiroshi Ishiguro
Date: 2017-02-28 00:00:00
Tasks: Deep Attention, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

For a safe, natural and effective human-robot social interaction, it is
essential to develop a system that allows a robot to demonstrate the
perceivable responsive behaviors to complex human behaviors. We introduce the
Multimodal Deep Attention Recurrent Q-Network using which the robot exhibits
human-like social interaction skills after 14 days of interacting with people
in an uncontrolled real world. Each and every day during the 14 days, the
system gathered robot interaction experiences with people through a
hit-and-trial method and then trained the MDARQN on these experiences using
end-to-end reinforcement learning approach. The results of interaction based
learning indicate that the robot has learned to respond to complex human
behaviors in a perceivable and socially acceptable manner.