pwc-archive/papers-with-abstracts huggingface dataset:
Harmonizable mixture kernels with variational Fourier features
Authors: Zheyang Shen, Markus Heinonen, Samuel Kaski
Date: 2018-10-10 00:00:00
Tasks: Gaussian Processes

The expressive power of Gaussian processes depends heavily on the choice of
kernel. In this work we propose the novel harmonizable mixture kernel (HMK), a
family of expressive, interpretable, non-stationary kernels derived from
mixture models on the generalized spectral representation. As a theoretically
sound treatment of non-stationary kernels, HMK supports harmonizable
covariances, a wide subset of kernels including all stationary and many
non-stationary covariances. We also propose variational Fourier features, an
inter-domain sparse GP inference framework that offers a representative set of
'inducing frequencies'. We show that harmonizable mixture kernels interpolate
between local patterns, and that variational Fourier features offers a robust
kernel learning framework for the new kernel family.
Prediction of the Influence of Navigation Scan-path on Perceived Quality of Free-Viewpoint Videos
Authors: Suiyi Ling, Jesús Gutiérrez, Gu Ke, Patrick Le Callet
Date: 2018-10-10 00:00:00
Tasks: Benchmarking, Video Quality Assessment

Free-Viewpoint Video (FVV) systems allow the viewers to freely change the
viewpoints of the scene. In such systems, view synthesis and compression are
the two main sources of artifacts influencing the perceived quality. To assess
this influence, quality evaluation studies are often carried out using
conventional displays and generating predefined navigation trajectories
mimicking the possible movement of the viewers when exploring the content.
Nevertheless, as different trajectories may lead to different conclusions in
terms of visual quality when benchmarking the performance of the systems,
methods to identify critical trajectories are needed. This paper aims at
exploring the impact of exploration trajectories (defined as Hypothetical
Rendering Trajectories: HRT) on perceived quality of FVV subjectively and
objectively, providing two main contributions. Firstly, a subjective assessment
test including different HRTs was carried out and analyzed. The results
demonstrate and quantify the influence of HRT in the perceived quality.
Secondly, we propose a new objective video quality assessment measure to
objectively predict the impact of HRT. This measure, based on Sketch-Token
representation, models how the categories of the contours change spatially and
temporally from a higher semantic level. Performance in comparison with
existing quality metrics for FVV, highlight promising results for automatic
detection of most critical HRTs for the benchmark of immersive systems.
Response to Comment on "All-optical machine learning using diffractive deep neural networks"
Authors: Deniz Mengu, Yi Luo, Yair Rivenson, Xing Lin, Muhammed Veli, Aydogan Ozcan
Date: 2018-10-10 00:00:00
Tasks: All, BIG-bench Machine Learning, valid

In their Comment, Wei et al. (arXiv:1809.08360v1 [cs.LG]) claim that our
original interpretation of Diffractive Deep Neural Networks (D2NN) represent a
mischaracterization of the system due to linearity and passivity. In this
Response, we detail how this mischaracterization claim is unwarranted and
oblivious to several sections detailed in our original manuscript (Science,
DOI: 10.1126/science.aat8084) that specifically introduced and discussed
optical nonlinearities and reconfigurability of D2NNs, as part of our proposed
framework to enhance its performance. To further refute the mischaracterization
claim of Wei et al., we, once again, demonstrate the depth feature of optical
D2NNs by showing that multiple diffractive layers operating collectively within
a D2NN present additional degrees-of-freedom compared to a single diffractive
layer to achieve better classification accuracy, as well as improved output
signal contrast and diffraction efficiency as the number of diffractive layers
increase, showing the deepness of a D2NN, and its inherent depth advantage for
improved performance. In summary, the Comment by Wei et al. does not provide an
amendment to the original teachings of our original manuscript, and all of our
results, core conclusions and methodology of research reported in Science (DOI:
10.1126/science.aat8084) remain entirely valid.
Learning Deep Representations for Semantic Image Parsing: a Comprehensive Overview
Authors: Lili Huang, Jiefeng Peng, Ruimao Zhang, Guanbin Li, Liang Lin
Date: 2018-10-10 00:00:00
Tasks: Representation Learning, Segmentation, Semantic Segmentation

Semantic image parsing, which refers to the process of decomposing images
into semantic regions and constructing the structure representation of the
input, has recently aroused widespread interest in the field of computer
vision. The recent application of deep representation learning has driven this
field into a new stage of development. In this paper, we summarize three
aspects of the progress of research on semantic image parsing, i.e.,
category-level semantic segmentation, instance-level semantic segmentation, and
beyond segmentation. Specifically, we first review the general frameworks for
each task and introduce the relevant variants. The advantages and limitations
of each method are also discussed. Moreover, we present a comprehensive
comparison of different benchmark datasets and evaluation metrics. Finally, we
explore the future trends and challenges of semantic image parsing.
On the Approximation Properties of Random ReLU Features
Authors: Yitong Sun, Anna Gilbert, Ambuj Tewari
Date: 2018-10-10 00:00:00

We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features whose feature maps are of the form of nodes in neural networks. The universality result implies that the random ReLU features method is a universally consistent learning algorithm. We prove that despite the universality of the RKHS induced by the random ReLU features, composition of functions in it generates substantially more complicated functions that are harder to approximate than those functions simply in the RKHS. We also prove that such composite functions can be efficiently approximated by multi-layer ReLU networks with bounded weights. This depth separation result shows that the random ReLU features models suffer from the same weakness as that of shallow models. We show in experiments that the performance of random ReLU features is comparable to that of random Fourier features and, in general, has a lower computational cost. We also demonstrate that when the target function is the composite function as described in the depth separation theorem, 3-layer neural networks indeed outperform both random ReLU features and 2-layer neural networks.
Semi-supervised clustering for de-duplication
Authors: Shrinu Kushagra, Shai Ben-David, Ihab Ilyas
Date: 2018-10-10 00:00:00
Tasks: Clustering

Data de-duplication is the task of detecting multiple records that correspond
to the same real-world entity in a database. In this work, we view
de-duplication as a clustering problem where the goal is to put records
corresponding to the same physical entity in the same cluster and putting
records corresponding to different physical entities into different clusters.
  We introduce a framework which we call promise correlation clustering. Given
a complete graph $G$ with the edges labelled $0$ and $1$, the goal is to find a
clustering that minimizes the number of $0$ edges within a cluster plus the
number of $1$ edges across different clusters (or correlation loss). The
optimal clustering can also be viewed as a complete graph $G^*$ with edges
corresponding to points in the same cluster being labelled $0$ and other edges
being labelled $1$. Under the promise that the edge difference between $G$ and
$G^*$ is "small", we prove that finding the optimal clustering (or $G^*$) is
still NP-Hard. [Ashtiani et. al, 2016] introduced the framework of
semi-supervised clustering, where the learning algorithm has access to an
oracle, which answers whether two points belong to the same or different
clusters. We further prove that even with access to a same-cluster oracle, the
promise version is NP-Hard as long as the number queries to the oracle is not
too large ($o(n)$ where $n$ is the number of vertices).
  Given these negative results, we consider a restricted version of correlation
clustering. As before, the goal is to find a clustering that minimizes the
correlation loss. However, we restrict ourselves to a given class $\mathcal F$
of clusterings. We offer a semi-supervised algorithmic approach to solve the
restricted variant with success guarantees.
Properly-weighted graph Laplacian for semi-supervised learning
Authors: Jeff Calder, Dejan Slepcev
Date: 2018-10-10 00:00:00

The performance of traditional graph Laplacian methods for semi-supervised
learning degrades substantially as the ratio of labeled to unlabeled data
decreases, due to a degeneracy in the graph Laplacian. Several approaches have
been proposed recently to address this, however we show that some of them
remain ill-posed in the large-data limit.
  In this paper, we show a way to correctly set the weights in Laplacian
regularization so that the estimator remains well posed and stable in the
large-sample limit. We prove that our semi-supervised learning algorithm
converges, in the infinite sample size limit, to the smooth solution of a
continuum variational problem that attains the labeled values continuously. Our
method is fast and easy to implement.
Combining Bayesian Optimization and Lipschitz Optimization
Authors: Mohamed Osama Ahmed, Sharan Vaswani, Mark Schmidt
Date: 2018-10-10 00:00:00
Tasks: Bayesian Optimization, global-optimization, Thompson Sampling

Bayesian optimization and Lipschitz optimization have developed alternative techniques for optimizing black-box functions. They each exploit a different form of prior about the function. In this work, we explore strategies to combine these techniques for better global optimization. In particular, we propose ways to use the Lipschitz continuity assumption within traditional BO algorithms, which we call Lipschitz Bayesian optimization (LBO). This approach does not increase the asymptotic runtime and in some cases drastically improves the performance (while in the worst-case the performance is similar). Indeed, in a particular setting, we prove that using the Lipschitz information yields the same or a better bound on the regret compared to using Bayesian optimization on its own. Moreover, we propose a simple heuristics to estimate the Lipschitz constant, and prove that a growing estimate of the Lipschitz constant is in some sense ``harmless''. Our experiments on 15 datasets with 4 acquisition functions show that in the worst case LBO performs similar to the underlying BO method while in some cases it performs substantially better. Thompson sampling in particular typically saw drastic improvements (as the Lipschitz information corrected for its well-known ``over-exploration'' phenomenon) and its LBO variant often outperformed other acquisition functions.
Task Runtime Prediction in Scientific Workflows Using an Online Incremental Learning Approach
Authors: Muhammad H. Hilman, Maria A. Rodriguez, Rajkumar Buyya
Date: 2018-10-10 00:00:00
Tasks: CPU, Incremental Learning, Management, regression, Scheduling, Time Series Analysis

Many algorithms in workflow scheduling and resource provisioning rely on the
performance estimation of tasks to produce a scheduling plan. A profiler that
is capable of modeling the execution of tasks and predicting their runtime
accurately, therefore, becomes an essential part of any Workflow Management
System (WMS). With the emergence of multi-tenant Workflow as a Service (WaaS)
platforms that use clouds for deploying scientific workflows, task runtime
prediction becomes more challenging because it requires the processing of a
significant amount of data in a near real-time scenario while dealing with the
performance variability of cloud resources. Hence, relying on methods such as
profiling tasks' execution data using basic statistical description (e.g.,
mean, standard deviation) or batch offline regression techniques to estimate
the runtime may not be suitable for such environments. In this paper, we
propose an online incremental learning approach to predict the runtime of tasks
in scientific workflows in clouds. To improve the performance of the
predictions, we harness fine-grained resources monitoring data in the form of
time-series records of CPU utilization, memory usage, and I/O activities that
are reflecting the unique characteristics of a task's execution. We compare our
solution to a state-of-the-art approach that exploits the resources monitoring
data based on regression machine learning technique. From our experiments, the
proposed strategy improves the performance, in terms of the error, up to
29.89%, compared to the state-of-the-art solutions.
Complementary-Label Learning for Arbitrary Losses and Models
Authors: Takashi Ishida, Gang Niu, Aditya Krishna Menon, Masashi Sugiyama
Date: 2018-10-10 00:00:00
Proceeding: Proceedings of the 36th International Conference on Machine Learning, 2019 6
Tasks: General Classification, Image Classification

In contrast to the standard classification paradigm where the true class is given to each training pattern, complementary-label learning only uses training patterns each equipped with a complementary label, which only specifies one of the classes that the pattern does not belong to. The goal of this paper is to derive a novel framework of complementary-label learning with an unbiased estimator of the classification risk, for arbitrary losses and models---all existing methods have failed to achieve this goal. Not only is this beneficial for the learning stage, it also makes model/hyper-parameter selection (through cross-validation) possible without the need of any ordinarily labeled validation data, while using any linear/non-linear models or convex/non-convex loss functions. We further improve the risk estimator by a non-negative correction and gradient ascent trick, and demonstrate its superiority through experiments.
Least Squares Normalized Cross Correlation
Authors: Oliver J. Woodford
Date: 2018-10-10 00:00:00

Direct methods are widely used for alignment of models to images, due to their accuracy, since they minimize errors in the domain of measurement noise. They have leveraged least squares minimizations, for simple, efficient, variational optimization, since the seminal 1981 work of Lucas & Kanade, and normalized cross correlation (NCC), for robustness to intensity variations, since at least 1972. Despite the complementary benefits of these two well known methods, they have not been effectively combined to address local variations in intensity. Many ad-hoc NCC frameworks, sub-optimal least squares methods and image transformation approaches have thus been proposed instead, each with their own limitations. This work shows that a least squares optimization of NCC without approximation is not only possible, but straightforward and efficient. A robust, locally normalized formulation is introduced to mitigate local intensity variations and partial occlusions. Finally, sparse features with oriented patches are proposed for further efficiency. The resulting framework is simple to implement, computationally efficient and robust to local intensity variations. It is evaluated on the image alignment problem, showing improvements in both convergence rate and computation time over existing lighting invariant methods.
Convex Functions in ACL2(r)
Authors: Carl Kwan, Mark R. Greenstreet
Date: 2018-10-10 00:00:00

This paper builds upon our prior formalisation of R^n in ACL2(r) by
presenting a set of theorems for reasoning about convex functions. This is a
demonstration of the higher-dimensional analytical reasoning possible in our
metric space formalisation of R^n. Among the introduced theorems is a set of
equivalent conditions for convex functions with Lipschitz continuous gradients
from Yurii Nesterov's classic text on convex optimisation. To the best of our
knowledge a full proof of the theorem has yet to be published in a single piece
of literature. We also explore "proof engineering" issues, such as how to state
Nesterov's theorem in a manner that is both clear and useful.
Real Vector Spaces and the Cauchy-Schwarz Inequality in ACL2(r)
Authors: Carl Kwan, Mark R. Greenstreet
Date: 2018-10-10 00:00:00

We present a mechanical proof of the Cauchy-Schwarz inequality in ACL2(r) and
a formalisation of the necessary mathematics to undertake such a proof. This
includes the formalisation of $\mathbb{R}^n$ as an inner product space. We also
provide an application of Cauchy-Schwarz by formalising $\mathbb R^n$ as a
metric space and exhibiting continuity for some simple functions $\mathbb
R^n\to\mathbb R$. The Cauchy-Schwarz inequality relates the magnitude of a
vector to its projection (or inner product) with another: \[|\langle
u,v\rangle| \leq \|u\| \|v\|\] with equality iff the vectors are linearly
dependent. It finds frequent use in many branches of mathematics including
linear algebra, real analysis, functional analysis, probability, etc. Indeed,
the inequality is considered to be among "The Hundred Greatest Theorems" and is
listed in the "Formalizing 100 Theorems" project. To the best of our knowledge,
our formalisation is the first published proof using ACL2(r) or any other
first-order theorem prover.
Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation
Authors: Micah J Sheller, G. Anthony Reina, Brandon Edwards, Jason Martin, Spyridon Bakas
Date: 2018-10-10 00:00:00
Tasks: Brain Tumor Segmentation, Deep Learning, Federated Learning, Segmentation, Semantic Segmentation, Tumor Segmentation

Deep learning models for semantic segmentation of images require large
amounts of data. In the medical imaging domain, acquiring sufficient data is a
significant challenge. Labeling medical image data requires expert knowledge.
Collaboration between institutions could address this challenge, but sharing
medical data to a centralized location faces various legal, privacy, technical,
and data-ownership challenges, especially among international institutions. In
this study, we introduce the first use of federated learning for
multi-institutional collaboration, enabling deep learning modeling without
sharing patient data. Our quantitative results demonstrate that the performance
of federated semantic segmentation models (Dice=0.852) on multimodal brain
scans is similar to that of models trained by sharing data (Dice=0.862). We
compare federated learning with two alternative collaborative learning methods
and find that they fail to match the performance of federated learning.
Batch Active Preference-Based Learning of Reward Functions
Authors: Erdem Biyik, Dorsa Sadigh
Date: 2018-10-10 00:00:00
Tasks: Active Learning

Data generation and labeling are usually an expensive part of learning for
robotics. While active learning methods are commonly used to tackle the former
problem, preference-based learning is a concept that attempts to solve the
latter by querying users with preference questions. In this paper, we will
develop a new algorithm, batch active preference-based learning, that enables
efficient learning of reward functions using as few data samples as possible
while still having short query generation times. We introduce several
approximations to the batch active learning problem, and provide theoretical
guarantees for the convergence of our algorithms. Finally, we present our
experimental results for a variety of robotics tasks in simulation. Our results
suggest that our batch active learning algorithm requires only a few queries
that are computed in a short amount of time. We then showcase our algorithm in
a study to learn human users' preferences.
Decipherment of Historical Manuscript Images
Authors: Xusen Yin, Nada Aldarrab, Beáta Megyesi, Kevin Knight
Date: 2018-10-09 00:00:00
Tasks: Clustering, Decipherment, Image Clustering

European libraries and archives are filled with enciphered manuscripts from the early modern period. These include military and diplomatic correspondence, records of secret societies, private letters, and so on. Although they are enciphered with classical cryptographic algorithms, their contents are unavailable to working historians. We therefore attack the problem of automatically converting cipher manuscript images into plaintext. We develop unsupervised models for character segmentation, character-image clustering, and decipherment of cluster sequences. We experiment with both pipelined and joint models, and we give empirical results for multiple ciphers.
A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models
Authors: Ying Nian Wu, Ruiqi Gao, Tian Han, Song-Chun Zhu
Date: 2018-10-09 00:00:00
Tasks: Descriptive

The pattern theory of Grenander is a mathematical framework where patterns
are represented by probability models on random variables of algebraic
structures. In this paper, we review three families of probability models,
namely, the discriminative models, the descriptive models, and the generative
models. A discriminative model is in the form of a classifier. It specifies the
conditional probability of the class label given the input signal. A
descriptive model specifies the probability distribution of the signal, based
on an energy function defined on the signal. A generative model assumes that
the signal is generated by some latent variables via a transformation. We shall
review these models within a common framework and explore their connections. We
shall also review the recent developments that take advantage of the high
approximation capacities of deep neural networks.
Inter-Scanner Harmonization of High Angular Resolution DW-MRI using Null Space Deep Learning
Authors: Vishwesh Nath, Prasanna Parvathaneni, Colin B. Hansen, Allison E. Hainline, Camilo Bermudez, Samuel Remedios, Justin A. Blaber, Kurt G. Schilling, Ilwoo Lyu, Vaibhav Janve, Yurui Gao, Iwona Stepniewska, Baxter P. Rogers, Allen T. Newton, L. Taylor Davis, Jeff Luci, Adam W. Anderson, Bennett A. Landman
Date: 2018-10-09 00:00:00

Diffusion-weighted magnetic resonance imaging (DW-MRI) allows for
non-invasive imaging of the local fiber architecture of the human brain at a
millimetric scale. Multiple classical approaches have been proposed to detect
both single (e.g., tensors) and multiple (e.g., constrained spherical
deconvolution, CSD) fiber population orientations per voxel. However, existing
techniques generally exhibit low reproducibility across MRI scanners. Herein,
we propose a data-driven tech-nique using a neural network design which
exploits two categories of data. First, training data were acquired on three
squirrel monkey brains using ex-vivo DW-MRI and histology of the brain. Second,
repeated scans of human subjects were acquired on two different scanners to
augment the learning of the network pro-posed. To use these data, we propose a
new network architecture, the null space deep network (NSDN), to simultaneously
learn on traditional observed/truth pairs (e.g., MRI-histology voxels) along
with repeated observations without a known truth (e.g., scan-rescan MRI). The
NSDN was tested on twenty percent of the histology voxels that were kept
completely blind to the network. NSDN significantly improved absolute
performance relative to histology by 3.87% over CSD and 1.42% over a recently
proposed deep neural network approach. More-over, it improved reproducibility
on the paired data by 21.19% over CSD and 10.09% over a recently proposed deep
approach. Finally, NSDN improved gen-eralizability of the model to a third in
vivo human scanner (which was not used in training) by 16.08% over CSD and
10.41% over a recently proposed deep learn-ing approach. This work suggests
that data-driven approaches for local fiber re-construction are more
reproducible, informative and precise and offers a novel, practical method for
determining these models.