pwc-archive/papers-with-abstracts huggingface dataset:
Novel quantitative indicators of digital ophthalmoscopy image quality
Authors: Chris von Csefalvay
Date: 2019-03-07 00:00:00
Tasks: Image Quality Assessment, Small Data Image Classification

With the advent of smartphone indirect ophthalmoscopy, teleophthalmology -
the use of specialist ophthalmology assets at a distance from the patient - has
experienced a breakthrough, promising enormous benefits especially for
healthcare in distant, inaccessible or opthalmologically underserved areas,
where specialists are either unavailable or too few in number. However,
accurate teleophthalmology requires high-quality ophthalmoscopic imagery. This
paper considers three feature families - statistical metrics, gradient-based
metrics and wavelet transform coefficient derived indicators - as possible
metrics to identify unsharp or blurry images. By using standard machine
learning techniques, the suitability of these features for image quality
assessment is confirmed, albeit on a rather small data set. With the increased
availability and decreasing cost of digital ophthalmoscopy on one hand and the
increased prevalence of diabetic retinopathy worldwide on the other, creating
tools that can determine whether an image is likely to be diagnostically
suitable can play a significant role in accelerating and streamlining the
teleophthalmology process. This paper highlights the need for more research in
this area, including the compilation of a diverse database of ophthalmoscopic
imagery, annotated with quality markers, to train the Point of Acquisition
error detection algorithms of the future.
Stratified Labeling for Surface Consistent Parallax Correction and Occlusion Completion
Authors: Jie Chen, Lap-Pui Chau, Junhui Hou
Date: 2019-03-07 00:00:00
Tasks: Generative Adversarial Network, Novel View Synthesis

The light field faithfully records the spatial and angular configurations of
the scene, which facilitates a wide range of imaging possibilities. In this
work, we propose an LF synthesis algorithm which renders high quality novel LF
views far outside the range of angular baselines of the given references. A
stratified synthesis strategy is adopted which parses the scene content based
on stratified disparity layers and across a varying range of spatial
granularities. Such a stratified methodology proves to help preserve scene
structures over large perspective shifts, and it provides informative clues for
inferring the textures of occluded regions. A generative-adversarial network
model is further adopted for parallax correction and occlusion completion
conditioned on the stratified synthesis features. Experiments show that our
proposed model can provide more reliable novel view synthesis quality at large
baseline extension ratios. Over 3dB quality improvement has been achieved
against state-of-the-art LF view synthesis algorithms.
A Novel Neural Network Structure Constructed according to Logical Relations
Authors: Wang Gang
Date: 2019-03-07 00:00:00

To solve more complex things, computer systems becomes more and more complex.
It becomes harder to be handled manually for various conditions and unknown new
conditions in advance. This situation urgently requires the development of
computer technology of automatic judgement and decision according to various
conditions. Current ANN (Artificial Neural Network) models are good at
perceptual intelligence while they are not good at cognitive intelligence such
as logical representation, making them not deal with the above situation well.
Therefore, researchers have tried to design novel models so as to represent and
store logical relations into the neural network structures, the type of which
is called KBNN (Knowledge-Based Neural Network). In this type models, the
neurons and links are designed specific for logical relation representation,
and the neural network structures are constructed according to logical
relations, allowing us to construct automatically the rule libraries of expert
systems. In this paper, the further improvement is made based on KBNN by
redesigning the neurons and links. This improvement can make neurons solely for
representing things while making links solely for representing logical
relations between things, and thus no extra logical neurons are needed.
Moreover, the related construction and adjustment methods of the neural network
structure are also designed based on the redesigned neurons and links, making
the neural network structure dynamically constructed and adjusted according to
the logical relations. The probabilistic mechanism for the weight adjustment
can make the neural network model further represent logical relations in the
uncertainty.
Synthetic Human Model Dataset for Skeleton Driven Non-rigid Motion Tracking and 3D Reconstruction
Authors: Shafeeq Elanattil, Peyman Moghadam
Date: 2019-03-07 00:00:00
Tasks: 3D Human Reconstruction, 3D Reconstruction

We introduce a synthetic dataset for evaluating non-rigid 3D human
reconstruction based on conventional RGB-D cameras. The dataset consist of
seven motion sequences of a single human model. For each motion sequence
per-frame ground truth geometry and ground truth skeleton are given. The
dataset also contains skinning weights of the human model. More information
about the dataset can be found at:
https://research.csiro.au/robotics/our-work/databases/synthetic-human-model-dataset/
Discovering Visual Patterns in Art Collections with Spatially-consistent Feature Learning
Authors: Xi Shen, Alexei A. Efros, Mathieu Aubry
Date: 2019-03-07 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Retrieval, Self-Supervised Learning

Our goal in this paper is to discover near duplicate patterns in large
collections of artworks. This is harder than standard instance mining due to
differences in the artistic media (oil, pastel, drawing, etc), and
imperfections inherent in the copying process. The key technical insight is to
adapt a standard deep feature to this task by fine-tuning it on the specific
art collection using self-supervised learning. More specifically, spatial
consistency between neighbouring feature matches is used as supervisory
fine-tuning signal. The adapted feature leads to more accurate style-invariant
matching, and can be used with a standard discovery approach, based on
geometric verification, to identify duplicate patterns in the dataset. The
approach is evaluated on several different datasets and shows surprisingly good
qualitative discovery results. For quantitative evaluation of the method, we
annotated 273 near duplicate details in a dataset of 1587 artworks attributed
to Jan Brueghel and his workshop. Beyond artwork, we also demonstrate
improvement on localization on the Oxford5K photo dataset as well as on
historical photograph localization on the Large Time Lags Location (LTLL)
dataset.
A Rank-1 Sketch for Matrix Multiplicative Weights
Authors: Yair Carmon, John C. Duchi, Aaron Sidford, Kevin Tian
Date: 2019-03-07 00:00:00
Tasks: Form

We show that a simple randomized sketch of the matrix multiplicative weight (MMW) update enjoys (in expectation) the same regret bounds as MMW, up to a small constant factor. Unlike MMW, where every step requires full matrix exponentiation, our steps require only a single product of the form $e^A b$, which the Lanczos method approximates efficiently. Our key technique is to view the sketch as a $\textit{randomized mirror projection}$, and perform mirror descent analysis on the $\textit{expected projection}$. Our sketch solves the online eigenvector problem, improving the best known complexity bounds by $\Omega(\log^5 n)$. We also apply this sketch to semidefinite programming in saddle-point form, yielding a simple primal-dual scheme with guarantees matching the best in the literature.
Creation and Evaluation of Datasets for Distributional Semantics Tasks in the Digital Humanities Domain
Authors: Gerhard Wohlgenannt, Ariadna Barinova, Dmitry Ilvovsky, Ekaterina Chernyak
Date: 2019-03-07 00:00:00
Tasks: Word Embeddings, Word Similarity

Word embeddings are already well studied in the general domain, usually
trained on large text corpora, and have been evaluated for example on word
similarity and analogy tasks, but also as an input to downstream NLP processes.
In contrast, in this work we explore the suitability of word embedding
technologies in the specialized digital humanities domain. After training
embedding models of various types on two popular fantasy novel book series, we
evaluate their performance on two task types: term analogies, and word
intrusion. To this end, we manually construct test datasets with domain
experts. Among the contributions are the evaluation of various word embedding
techniques on the different task types, with the findings that even embeddings
trained on small corpora perform well for example on the word intrusion task.
Furthermore, we provide extensive and high-quality datasets in digital
humanities for further investigation, as well as the implementation to easily
reproduce or extend the experiments.
Understanding Ancient Coin Images
Authors: Jessica Cooper, Ognjen Arandjelovic
Date: 2019-03-07 00:00:00

In recent years, a range of problems within the broad umbrella of automatic,
computer vision based analysis of ancient coins has been attracting an
increasing amount of attention. Notwithstanding this research effort, the
results achieved by the state of the art in the published literature remain
poor and far from sufficiently well performing for any practical purpose. In
the present paper we present a series of contributions which we believe will
benefit the interested community. Firstly, we explain that the approach of
visual matching of coins, universally adopted in all existing published papers
on the topic, is not of practical interest because the number of ancient coin
types exceeds by far the number of those types which have been imaged, be it in
digital form (e.g. online) or otherwise (traditional film, in print, etc.).
Rather, we argue that the focus should be on the understanding of the semantic
content of coins. Hence, we describe a novel method which uses real-world
multimodal input to extract and associate semantic concepts with the correct
coin images and then using a novel convolutional neural network learn the
appearance of these concepts. Empirical evidence on a real-world and by far the
largest data set of ancient coins, we demonstrate highly promising results.
On Convergence Rate of the Gaussian Belief Propagation Algorithm for Markov Networks
Authors: Zhaorong Zhang, Minyue Fu
Date: 2019-03-06 00:00:00

Gaussian Belief Propagation (BP) algorithm is one of the most important
distributed algorithms in signal processing and statistical learning involving
Markov networks. It is well known that the algorithm correctly computes
marginal density functions from a high dimensional joint density function over
a Markov network in a finite number of iterations when the underlying Gaussian
graph is acyclic. It is also known more recently that the algorithm produces
correct marginal means asymptotically for cyclic Gaussian graphs under the
condition of walk summability. This paper extends this convergence result
further by showing that the convergence is exponential under the walk
summability condition, and provides a simple bound for the convergence rate.
Fast Parallel Algorithms for Statistical Subset Selection Problems
Authors: Sharon Qian, Yaron Singer
Date: 2019-03-06 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Combinatorial Optimization, Experimental Design, feature selection

In this paper, we propose a new framework for designing fast parallel algorithms for fundamental statistical subset selection tasks that include feature selection and experimental design. Such tasks are known to be weakly submodular and are amenable to optimization via the standard greedy algorithm. Despite its desirable approximation guarantees, the greedy algorithm is inherently sequential and in the worst case, its parallel runtime is linear in the size of the data. Recently, there has been a surge of interest in a parallel optimization technique called adaptive sampling which produces solutions with desirable approximation guarantees for submodular maximization in exponentially faster parallel runtime. Unfortunately, we show that for general weakly submodular functions such accelerations are impossible. The major contribution in this paper is a novel relaxation of submodularity which we call differential submodularity. We first prove that differential submodularity characterizes objectives like feature selection and experimental design. We then design an adaptive sampling algorithm for differentially submodular functions whose parallel runtime is logarithmic in the size of the data and achieves strong approximation guarantees. Through experiments, we show the algorithm's performance is competitive with state-of-the-art methods and obtains dramatic speedups for feature selection and experimental design problems.
Multi-Instance Learning for End-to-End Knowledge Base Question Answering
Authors: Mengxi Wei, Yifan He, Qiong Zhang, Luo Si
Date: 2019-03-06 00:00:00
Tasks: Knowledge Base Question Answering, Multiple Instance Learning, Question Answering

End-to-end training has been a popular approach for knowledge base question
answering (KBQA). However, real world applications often contain answers of
varied quality for users' questions. It is not appropriate to treat all
available answers of a user question equally.
  This paper proposes a novel approach based on multiple instance learning to
address the problem of noisy answers by exploring consensus among answers to
the same question in training end-to-end KBQA models. In particular, the QA
pairs are organized into bags with dynamic instance selection and different
options of instance weighting. Curriculum learning is utilized to select
instance bags during training. On the public CQA dataset, the new method
significantly improves both entity accuracy and the Rouge-L score over a
state-of-the-art end-to-end KBQA baseline.
Learning Graphs from Noisy Epidemic Cascades
Authors: Jessica Hoffmann, Constantine Caramanis
Date: 2019-03-06 00:00:00

We consider the problem of learning the weighted edges of a graph by observing the noisy times of infection for multiple epidemic cascades on this graph. Past work has considered this problem when the cascade information, i.e., infection times, are known exactly. Though the noisy setting is well motivated by many epidemic processes (e.g., most human epidemics), to the best of our knowledge, very little is known about when it is solvable. Previous work on the no-noise setting critically uses the ordering information. If noise can reverse this -- a node's reported (noisy) infection time comes after the reported infection time of some node it infected -- then we are unable to see how previous results can be extended. We therefore tackle two versions of the noisy setting: the limited-noise setting, where we know noisy times of infections, and the extreme-noise setting, in which we only know whether or not a node was infected. We provide a polynomial time algorithm for recovering the structure of bidirectional trees in the extreme-noise setting, and show our algorithm almost matches lower bounds established in the no-noise setting, and hence is optimal up to log-factors. We extend our results for general degree-bounded graphs, where again we show that our (poly-time) algorithm can recover the structure of the graph with optimal sample complexity. We also provide the first efficient algorithm to learn the weights of the bidirectional tree in the limited-noise setting. Finally, we give a polynomial time algorithm for learning the weights of general bounded-degree graphs in the limited-noise setting. This algorithm extends to general graphs (at the price of exponential running time), proving the problem is solvable in the general case. All our algorithms work for any noise distribution, without any restriction on the variance.
Continual Learning Using World Models for Pseudo-Rehearsal
Authors: Nicholas Ketz, Soheil Kolouri, Praveen Pilly
Date: 2019-03-06 00:00:00
Tasks: Atari Games, Continual Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The utility of learning a dynamics/world model of the environment in reinforcement learning has been shown in a many ways. When using neural networks, however, these models suffer catastrophic forgetting when learned in a lifelong or continual fashion. Current solutions to the continual learning problem require experience to be segmented and labeled as discrete tasks, however, in continuous experience it is generally unclear what a sufficient segmentation of tasks would be. Here we propose a method to continually learn these internal world models through the interleaving of internally generated episodes of past experiences (i.e., pseudo-rehearsal). We show this method can sequentially learn unsupervised temporal prediction, without task labels, in a disparate set of Atari games. Empirically, this interleaving of the internally generated rollouts with the external environment's observations leads to a consistent reduction in temporal prediction loss compared to non-interleaved learning and is preserved over repeated random exposures to various tasks. Similarly, using a network distillation approach, we show that modern policy gradient based reinforcement learning algorithms can use this internal model to continually learn to optimize reward based on the world model's representation of the environment.
A Character-Level Approach to the Text Normalization Problem Based on a New Causal Encoder
Authors: Adrián Javaloy Bornás, Ginés García Mateos
Date: 2019-03-06 00:00:00
Tasks: Decoder, Text Normalization

Text normalization is a ubiquitous process that appears as the first step of
many Natural Language Processing problems. However, previous Deep Learning
approaches have suffered from so-called silly errors, which are undetectable on
unsupervised frameworks, making those models unsuitable for deployment. In this
work, we make use of an attention-based encoder-decoder architecture that
overcomes these undetectable errors by using a fine-grained character-level
approach rather than a word-level one. Furthermore, our new general-purpose
encoder based on causal convolutions, called Causal Feature Extractor (CFE), is
introduced and compared to other common encoders. The experimental results show
the feasibility of this encoder, which leverages the attention mechanisms the
most and obtains better results in terms of accuracy, number of parameters and
convergence time. While our method results in a slightly worse initial accuracy
(92.74%), errors can be automatically detected and, thus, more readily solved,
obtaining a more robust model for deployment. Furthermore, there is still
plenty of room for future improvements that will push even further these
advantages.
Generative Graph Convolutional Network for Growing Graphs
Authors: Da Xu, Chuanwei Ruan, Kamiya Motwani, Evren Korpeoglu, Sushant Kumar, Kannan Achan
Date: 2019-03-06 00:00:00
Tasks: Graph Generation, Graph Reconstruction, Recommendation Systems

Modeling generative process of growing graphs has wide applications in social
networks and recommendation systems, where cold start problem leads to new
nodes isolated from existing graph. Despite the emerging literature in learning
graph representation and graph generation, most of them can not handle isolated
new nodes without nontrivial modifications. The challenge arises due to the
fact that learning to generate representations for nodes in observed graph
relies heavily on topological features, whereas for new nodes only node
attributes are available. Here we propose a unified generative graph
convolutional network that learns node representations for all nodes adaptively
in a generative model framework, by sampling graph generation sequences
constructed from observed graph data. We optimize over a variational lower
bound that consists of a graph reconstruction term and an adaptive
Kullback-Leibler divergence regularization term. We demonstrate the superior
performance of our approach on several benchmark citation network datasets.
IMEXnet: A Forward Stable Deep Neural Network
Authors: Eldad Haber, Keegan Lensink, Eran Treister, Lars Ruthotto
Date: 2019-03-06 00:00:00
Tasks: Semantic Segmentation

Deep convolutional neural networks have revolutionized many machine learning and computer vision tasks, however, some remaining key challenges limit their wider use. These challenges include improving the network's robustness to perturbations of the input image and the limited ``field of view'' of convolution operators. We introduce the IMEXnet that addresses these challenges by adapting semi-implicit methods for partial differential equations. Compared to similar explicit networks, such as residual networks, our network is more stable, which has recently shown to reduce the sensitivity to small changes in the input features and improve generalization. The addition of an implicit step connects all pixels in each channel of the image and therefore addresses the field of view problem while still being comparable to standard convolutions in terms of the number of parameters and computational complexity. We also present a new dataset for semantic segmentation and demonstrate the effectiveness of our architecture using the NYU Depth dataset.
Deep Random Splines for Point Process Intensity Estimation of Neural Population Data
Authors: Gabriel Loaiza-Ganem, Sean M. Perkins, Karen E. Schroeder, Mark M. Churchland, John P. Cunningham
Date: 2019-03-06 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Dimensionality Reduction, Gaussian Processes, Point Processes

Gaussian processes are the leading class of distributions on random functions, but they suffer from well known issues including difficulty scaling and inflexibility with respect to certain shape constraints (such as nonnegativity). Here we propose Deep Random Splines, a flexible class of random functions obtained by transforming Gaussian noise through a deep neural network whose output are the parameters of a spline. Unlike Gaussian processes, Deep Random Splines allow us to readily enforce shape constraints while inheriting the richness and tractability of deep generative models. We also present an observational model for point process data which uses Deep Random Splines to model the intensity function of each point process and apply it to neural population data to obtain a low-dimensional representation of spiking activity. Inference is performed via a variational autoencoder that uses a novel recurrent encoder architecture that can handle multiple point processes as input. We use a newly collected dataset where a primate completes a pedaling task, and observe better dimensionality reduction with our model than with competing alternatives.
Mean-field Analysis of Batch Normalization
Authors: Mingwei Wei, James Stokes, David J. Schwab
Date: 2019-03-06 00:00:00
Proceeding: ICLR 2019 5

Batch Normalization (BatchNorm) is an extremely useful component of modern
neural network architectures, enabling optimization using higher learning rates
and achieving faster convergence. In this paper, we use mean-field theory to
analytically quantify the impact of BatchNorm on the geometry of the loss
landscape for multi-layer networks consisting of fully-connected and
convolutional layers. We show that it has a flattening effect on the loss
landscape, as quantified by the maximum eigenvalue of the Fisher Information
Matrix. These findings are then used to justify the use of larger learning
rates for networks that use BatchNorm, and we provide quantitative
characterization of the maximal allowable learning rate to ensure convergence.
Experiments support our theoretically predicted maximum learning rate, and
furthermore suggest that networks with smaller values of the BatchNorm
parameter achieve lower loss after the same number of epochs of training.