pwc-archive/papers-with-abstracts huggingface dataset:
NIMA: Neural Image Assessment
Authors: Hossein Talebi, Peyman Milanfar
Date: 2017-09-15 00:00:00
Tasks: Aesthetics Quality Assessment, Image Quality Assessment, Video Quality Assessment

Automatically learned quality assessment for images has recently become a hot
topic due to its usefulness in a wide variety of applications such as
evaluating image capture pipelines, storage techniques and sharing media.
Despite the subjective nature of this problem, most existing methods only
predict the mean opinion score provided by datasets such as AVA [1] and TID2013
[2]. Our approach differs from others in that we predict the distribution of
human opinion scores using a convolutional neural network. Our architecture
also has the advantage of being significantly simpler than other methods with
comparable performance. Our proposed approach relies on the success (and
retraining) of proven, state-of-the-art deep object recognition networks. Our
resulting network can be used to not only score images reliably and with high
correlation to human perception, but also to assist with adaptation and
optimization of photo editing/enhancement algorithms in a photographic
pipeline. All this is done without need for a "golden" reference image,
consequently allowing for single-image, semantic- and perceptually-aware,
no-reference quality assessment.
Universal approximations of invariant maps by neural networks
Authors: Dmitry Yarotsky
Date: 2018-04-26 00:00:00

We describe generalizations of the universal approximation theorem for neural
networks to maps invariant or equivariant with respect to linear
representations of groups. Our goal is to establish network-like computational
models that are both invariant/equivariant and provably complete in the sense
of their ability to approximate any continuous invariant/equivariant map. Our
contribution is three-fold. First, in the general case of compact groups we
propose a construction of a complete invariant/equivariant network using an
intermediate polynomial layer. We invoke classical theorems of Hilbert and Weyl
to justify and simplify this construction; in particular, we describe an
explicit complete ansatz for approximation of permutation-invariant maps.
Second, we consider groups of translations and prove several versions of the
universal approximation theorem for convolutional networks in the limit of
continuous signals on euclidean spaces. Finally, we consider 2D signal
transformations equivariant with respect to the group SE(2) of rigid euclidean
motions. In this case we introduce the "charge--conserving convnet" -- a
convnet-like computational model based on the decomposition of the feature
space into isotypic representations of SO(2). We prove this model to be a
universal approximator for continuous SE(2)--equivariant signal
transformations.
Distributed Differentially-Private Algorithms for Matrix and Tensor Factorization
Authors: Hafiz Imtiaz, Anand D. Sarwate
Date: 2018-04-26 00:00:00
Tasks: Privacy Preserving, Tensor Decomposition

In many signal processing and machine learning applications, datasets
containing private information are held at different locations, requiring the
development of distributed privacy-preserving algorithms. Tensor and matrix
factorizations are key components of many processing pipelines. In the
distributed setting, differentially private algorithms suffer because they
introduce noise to guarantee privacy. This paper designs new and improved
distributed and differentially private algorithms for two popular matrix and
tensor factorization methods: principal component analysis (PCA) and orthogonal
tensor decomposition (OTD). The new algorithms employ a correlated noise design
scheme to alleviate the effects of noise and can achieve the same noise level
as the centralized scenario. Experiments on synthetic and real data illustrate
the regimes in which the correlated noise allows performance matching with the
centralized setting, outperforming previous methods and demonstrating that
meaningful utility is possible while guaranteeing differential privacy.
Unsupervised Learning of Sequence Representations by Autoencoders
Authors: Wenjie Pei, David M. J. Tax
Date: 2018-04-03 00:00:00

Sequence data is challenging for machine learning approaches, because the
lengths of the sequences may vary between samples. In this paper, we present an
unsupervised learning model for sequence data, called the Integrated Sequence
Autoencoder (ISA), to learn a fixed-length vectorial representation by
minimizing the reconstruction error. Specifically, we propose to integrate two
classical mechanisms for sequence reconstruction which takes into account both
the global silhouette information and the local temporal dependencies.
Furthermore, we propose a stop feature that serves as a temporal stamp to guide
the reconstruction process, which results in a higher-quality representation.
The learned representation is able to effectively summarize not only the
apparent features, but also the underlying and high-level style information.
Take for example a speech sequence sample: our ISA model can not only recognize
the spoken text (apparent feature), but can also discriminate the speaker who
utters the audio (more high-level style). One promising application of the ISA
model is that it can be readily used in the semi-supervised learning scenario,
in which a large amount of unlabeled data is leveraged to extract high-quality
sequence representations and thus to improve the performance of the subsequent
supervised learning tasks on limited labeled data.
RoadTracer: Automatic Extraction of Road Networks from Aerial Images
Authors: Favyen Bastani, Songtao He, Sofiane Abbar, Mohammad Alizadeh, Hari Balakrishnan, Sanjay Chawla, Sam Madden, David DeWitt
Date: 2018-02-11 00:00:00
Proceeding: CVPR 2018 6
Tasks: Road Segmentation, Segmentation

Mapping road networks is currently both expensive and labor-intensive.
High-resolution aerial imagery provides a promising avenue to automatically
infer a road network. Prior work uses convolutional neural networks (CNNs) to
detect which pixels belong to a road (segmentation), and then uses complex
post-processing heuristics to infer graph connectivity. We show that these
segmentation methods have high error rates because noisy CNN outputs are
difficult to correct. We propose RoadTracer, a new method to automatically
construct accurate road network maps from aerial images. RoadTracer uses an
iterative search process guided by a CNN-based decision function to derive the
road network graph directly from the output of the CNN. We compare our approach
with a segmentation method on fifteen cities, and find that at a 5% error rate,
RoadTracer correctly captures 45% more junctions across these cities.
Semi-supervised Embedding in Attributed Networks with Outliers
Authors: Jiongqian Liang, Peter Jacobs, Jiankai Sun, Srinivasan Parthasarathy
Date: 2017-03-23 00:00:00
Tasks: Attribute

In this paper, we propose a novel framework, called Semi-supervised Embedding
in Attributed Networks with Outliers (SEANO), to learn a low-dimensional vector
representation that systematically captures the topological proximity,
attribute affinity and label similarity of vertices in a partially labeled
attributed network (PLAN). Our method is designed to work in both transductive
and inductive settings while explicitly alleviating noise effects from
outliers. Experimental results on various datasets drawn from the web, text and
image domains demonstrate the advantages of SEANO over state-of-the-art methods
in semi-supervised classification under transductive as well as inductive
settings. We also show that a subset of parameters in SEANO is interpretable as
outlier score and can significantly outperform baseline methods when applied
for detecting network outliers. Finally, we present the use of SEANO in a
challenging real-world setting -- flood mapping of satellite images and show
that it is able to outperform modern remote sensing algorithms for this task.
Modeling polypharmacy side effects with graph convolutional networks
Authors: Marinka Zitnik, Monica Agrawal, Jure Leskovec
Date: 2018-02-02 00:00:00
Tasks: Link Prediction

The use of drug combinations, termed polypharmacy, is common to treat
patients with complex diseases and co-existing conditions. However, a major
consequence of polypharmacy is a much higher risk of adverse side effects for
the patient. Polypharmacy side effects emerge because of drug-drug
interactions, in which activity of one drug may change if taken with another
drug. The knowledge of drug interactions is limited because these complex
relationships are rare, and are usually not observed in relatively small
clinical testing. Discovering polypharmacy side effects thus remains an
important challenge with significant implications for patient mortality. Here,
we present Decagon, an approach for modeling polypharmacy side effects. The
approach constructs a multimodal graph of protein-protein interactions,
drug-protein target interactions, and the polypharmacy side effects, which are
represented as drug-drug interactions, where each side effect is an edge of a
different type. Decagon is developed specifically to handle such multimodal
graphs with a large number of edge types. Our approach develops a new graph
convolutional neural network for multirelational link prediction in multimodal
networks. Decagon predicts the exact side effect, if any, through which a given
drug combination manifests clinically. Decagon accurately predicts polypharmacy
side effects, outperforming baselines by up to 69%. We find that it
automatically learns representations of side effects indicative of
co-occurrence of polypharmacy in patients. Furthermore, Decagon models
particularly well side effects with a strong molecular basis, while on
predominantly non-molecular side effects, it achieves good performance because
of effective sharing of model parameters across edge types. Decagon creates
opportunities to use large pharmacogenomic and patient data to flag and
prioritize side effects for follow-up analysis.
Adaptive Sensing for Learning Nonstationary Environment Models
Authors: Sahil Garg, Amarjeet Singh, Fabio Ramos
Date: 2018-04-26 00:00:00
Tasks: Gaussian Processes

Most environmental phenomena, such as wind profiles, ozone concentration and
sunlight distribution under a forest canopy, exhibit nonstationary dynamics
i.e. phenomenon variation change depending on the location and time of
occurrence. Non-stationary dynamics pose both theoretical and practical
challenges to statistical machine learning algorithms aiming to accurately
capture the complexities governing the evolution of such processes. In this
paper, we address the sampling aspects of the problem of learning nonstationary
spatio-temporal models, and propose an efficient yet simple algorithm - LISAL.
The core idea in LISAL is to learn two models using Gaussian processes (GPs)
wherein the first is a nonstationary GP directly modeling the phenomenon. The
second model uses a stationary GP representing a latent space corresponding to
changes in dynamics, or the nonstationarity characteristics of the first model.
LISAL involves adaptively sampling the latent space dynamics using information
theory quantities to reduce the computational cost during the learning phase.
The relevance of LISAL is extensively validated using multiple real world
datasets.
Synthesized Texture Quality Assessment via Multi-scale Spatial and Statistical Texture Attributes of Image and Gradient Magnitude Coefficients
Authors: S. Alireza Golestaneh, Lina Karam
Date: 2018-04-21 00:00:00

Perceptual quality assessment for synthesized textures is a challenging task.
In this paper, we propose a training-free reduced-reference (RR) objective
quality assessment method that quantifies the perceived quality of synthesized
textures. The proposed reduced-reference synthesized texture quality assessment
metric is based on measuring the spatial and statistical attributes of the
texture image using both image- and gradient-based wavelet coefficients at
multiple scales. Performance evaluations on two synthesized texture databases
demonstrate that our proposed RR synthesized texture quality metric
significantly outperforms both full-reference and RR state-of-the-art quality
metrics in predicting the perceived visual quality of the synthesized textures.
Network Transplanting
Authors: Quanshi Zhang, Yu Yang, Qian Yu, Ying Nian Wu
Date: 2018-04-26 00:00:00

This paper focuses on a new task, i.e., transplanting a
category-and-task-specific neural network to a generic, modular network without
strong supervision. We design an functionally interpretable structure for the
generic network. Like building LEGO blocks, we teach the generic network a new
category by directly transplanting the module corresponding to the category
from a pre-trained network with a few or even without sample annotations. Our
method incrementally adds new categories to the generic network but does not
affect representations of existing categories. In this way, our method breaks
the typical bottleneck of learning a net for massive tasks and categories, i.e.
the requirement of collecting samples for all tasks and categories at the same
time before the learning begins. Thus, we use a new distillation algorithm,
namely back-distillation, to overcome specific challenges of network
transplanting. Our method without training samples even outperformed the
baseline with 100 training samples.
Tensor Methods for Nonlinear Matrix Completion
Authors: Greg Ongie, Daniel Pimentel-Alarcón, Laura Balzano, Rebecca Willett, Robert D. Nowak
Date: 2018-04-26 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion

In the low-rank matrix completion (LRMC) problem, the low-rank assumption means that the columns (or rows) of the matrix to be completed are points on a low-dimensional linear algebraic variety. This paper extends this thinking to cases where the columns are points on a low-dimensional nonlinear algebraic variety, a problem we call Low Algebraic Dimension Matrix Completion (LADMC). Matrices whose columns belong to a union of subspaces are an important special case. We propose a LADMC algorithm that leverages existing LRMC methods on a tensorized representation of the data. For example, a second-order tensorized representation is formed by taking the Kronecker product of each column with itself, and we consider higher order tensorizations as well. This approach will succeed in many cases where traditional LRMC is guaranteed to fail because the data are low-rank in the tensorized representation but not in the original representation. We provide a formal mathematical justification for the success of our method. In particular, we give bounds of the rank of these data in the tensorized representation, and we prove sampling requirements to guarantee uniqueness of the solution. We also provide experimental results showing that the new approach outperforms existing state-of-the-art methods for matrix completion under a union of subspaces model.
Incremental 3D Line Segment Extraction from Semi-dense SLAM
Authors: Shida He, Xuebin Qin, Zichen Zhang, Martin Jagersand
Date: 2017-08-10 00:00:00
Tasks: Clustering, Simultaneous Localization and Mapping, Surface Reconstruction

Although semi-dense Simultaneous Localization and Mapping (SLAM) has been
becoming more popular over the last few years, there is a lack of efficient
methods for representing and processing their large scale point clouds. In this
paper, we propose using 3D line segments to simplify the point clouds generated
by semi-dense SLAM. Specifically, we present a novel incremental approach for
3D line segment extraction. This approach reduces a 3D line segment fitting
problem into two 2D line segment fitting problems and takes advantage of both
images and depth maps. In our method, 3D line segments are fitted incrementally
along detected edge segments via minimizing fitting errors on two planes. By
clustering the detected line segments, the resulting 3D representation of the
scene achieves a good balance between compactness and completeness. Our
experimental results show that the 3D line segments generated by our method are
highly accurate. As an application, we demonstrate that these line segments
greatly improve the quality of 3D surface reconstruction compared to a feature
point based baseline.
From Principal Subspaces to Principal Components with Linear Autoencoders
Authors: Elad Plaut
Date: 2018-04-26 00:00:00
Tasks: Dimensionality Reduction

The autoencoder is an effective unsupervised learning model which is widely
used in deep learning. It is well known that an autoencoder with a single
fully-connected hidden layer, a linear activation function and a squared error
cost function trains weights that span the same subspace as the one spanned by
the principal component loading vectors, but that they are not identical to the
loading vectors. In this paper, we show how to recover the loading vectors from
the autoencoder weights.
Experimenting with robotic intra-logistics domains
Authors: Martin Gebser, Philipp Obermeier, Thomas Otto, Torsten Schaub, Orkunt Sabuncu, Van Nguyen, Tran Cao Son
Date: 2018-04-26 00:00:00
Tasks: Benchmarking, valid

We introduce the asprilo [1] framework to facilitate experimental studies of
approaches addressing complex dynamic applications. For this purpose, we have
chosen the domain of robotic intra-logistics. This domain is not only highly
relevant in the context of today's fourth industrial revolution but it moreover
combines a multitude of challenging issues within a single uniform framework.
This includes multi-agent planning, reasoning about action, change, resources,
strategies, etc. In return, asprilo allows users to study alternative solutions
as regards effectiveness and scalability. Although asprilo relies on Answer Set
Programming and Python, it is readily usable by any system complying with its
fact-oriented interface format. This makes it attractive for benchmarking and
teaching well beyond logic programming. More precisely, asprilo consists of a
versatile benchmark generator, solution checker and visualizer as well as a
bunch of reference encodings featuring various ASP techniques. Importantly, the
visualizer's animation capabilities are indispensable for complex scenarios
like intra-logistics in order to inspect valid as well as invalid solution
candidates. Also, it allows for graphically editing benchmark layouts that can
be used as a basis for generating benchmark suites.
  [1] asprilo stands for Answer Set Programming for robotic intra-logistics
Temporal Answer Set Programming on Finite Traces
Authors: Pedro Cabalar, Roland Kaminski, Torsten Schaub, Anna Schuhmann
Date: 2018-04-26 00:00:00

In this paper, we introduce an alternative approach to Temporal Answer Set
Programming that relies on a variation of Temporal Equilibrium Logic (TEL) for
finite traces. This approach allows us to even out the expressiveness of TEL
over infinite traces with the computational capacity of (incremental) Answer
Set Programming (ASP). Also, we argue that finite traces are more natural when
reasoning about action and change. As a result, our approach is readily
implementable via multi-shot ASP systems and benefits from an extension of
ASP's full-fledged input language with temporal operators. This includes future
as well as past operators whose combination offers a rich temporal modeling
language. For computation, we identify the class of temporal logic programs and
prove that it constitutes a normal form for our approach. Finally, we outline
two implementations, a generic one and an extension of clingo.
Sparse Persistent RNNs: Squeezing Large Recurrent Networks On-Chip
Authors: Feiwen Zhu, Jeff Pool, Michael Andersch, Jeremy Appleyard, Fung Xie
Date: 2018-04-26 00:00:00
Proceeding: ICLR 2018 1
Tasks: GPU, NMT, speech-recognition, Speech Recognition

Recurrent Neural Networks (RNNs) are powerful tools for solving
sequence-based problems, but their efficacy and execution time are dependent on
the size of the network. Following recent work in simplifying these networks
with model pruning and a novel mapping of work onto GPUs, we design an
efficient implementation for sparse RNNs. We investigate several optimizations
and tradeoffs: Lamport timestamps, wide memory loads, and a bank-aware weight
layout. With these optimizations, we achieve speedups of over 6x over the next
best algorithm for a hidden layer of size 2304, batch size of 4, and a density
of 30%. Further, our technique allows for models of over 5x the size to fit on
a GPU for a speedup of 2x, enabling larger networks to help advance the
state-of-the-art. We perform case studies on NMT and speech recognition tasks
in the appendix, accelerating their recurrent layers by up to 3x.
A Comprehensive Survey on Bengali Phoneme Recognition
Authors: Sadia Tasnim Swarna, Shamim Ehsan, Md. Saiful Islam, Marium E Jannat
Date: 2017-01-27 00:00:00
Tasks: Automatic Phoneme Recognition, Phoneme Recognition, speech-recognition, Speech Recognition, Survey

Hidden Markov model based various phoneme recognition methods for Bengali
language is reviewed. Automatic phoneme recognition for Bengali language using
multilayer neural network is reviewed. Usefulness of multilayer neural network
over single layer neural network is discussed. Bangla phonetic feature table
construction and enhancement for Bengali speech recognition is also discussed.
Comparison among these methods is discussed.
Learning Light Field Reconstruction from a Single Coded Image
Authors: Anil Kumar Vadathya, Saikiran Cholleti, Gautham Ramajayam, Vijayalakshmi Kanchana, Kaushik Mitra
Date: 2018-01-20 00:00:00
Tasks: Dictionary Learning, Disparity Estimation

Light field imaging is a rich way of representing the 3D world around us.
However, due to limited sensor resolution capturing light field data inherently
poses spatio-angular resolution trade-off. In this paper, we propose a deep
learning based solution to tackle the resolution trade-off. Specifically, we
reconstruct full sensor resolution light field from a single coded image. We
propose to do this in three stages 1) reconstruction of center view from the
coded image 2) estimating disparity map from the coded image and center view 3)
warping center view using the disparity to generate light field. We propose
three neural networks for these stages. Our disparity estimation network is
trained in an unsupervised manner alleviating the need for ground truth
disparity. Our results demonstrate better recovery of parallax from the coded
image. Also, we get better results than dictionary learning based approaches
both qualitatively and quatitatively.
The loss landscape of overparameterized neural networks
Authors: Y. Cooper
Date: 2018-04-26 00:00:00
Proceeding: ICLR 2019 5

We explore some mathematical features of the loss landscape of
overparameterized neural networks. A priori one might imagine that the loss
function looks like a typical function from $\mathbb{R}^n$ to $\mathbb{R}$ - in
particular, nonconvex, with discrete global minima. In this paper, we prove
that in at least one important way, the loss function of an overparameterized
neural network does not look like a typical function. If a neural net has $n$
parameters and is trained on $d$ data points, with $n>d$, we show that the
locus $M$ of global minima of $L$ is usually not discrete, but rather an $n-d$
dimensional submanifold of $\mathbb{R}^n$. In practice, neural nets commonly
have orders of magnitude more parameters than data points, so this observation
implies that $M$ is typically a very high-dimensional subset of $\mathbb{R}^n$.