pwc-archive/papers-with-abstracts huggingface dataset:
R3Net: Random Weights, Rectifier Linear Units and Robustness for Artificial Neural Network
Authors: Arun Venkitaraman, Alireza M. Javid, Saikat Chatterjee
Date: 2018-03-12 00:00:00

We consider a neural network architecture with randomized features, a
sign-splitter, followed by rectified linear units (ReLU). We prove that our
architecture exhibits robustness to the input perturbation: the output feature
of the neural network exhibits a Lipschitz continuity in terms of the input
perturbation. We further show that the network output exhibits a discrimination
ability that inputs that are not arbitrarily close generate output vectors
which maintain distance between each other obeying a certain lower bound. This
ensures that two different inputs remain discriminable while contracting the
distance in the output feature space.
A Framework for Wasserstein-1-Type Metrics
Authors: Bernhard Schmitzer, Benedikt Wirth
Date: 2017-01-08 00:00:00
Tasks: Computational Efficiency, Vocal Bursts Type Prediction

We propose a unifying framework for generalising the Wasserstein-1 metric to
a discrepancy measure between nonnegative measures of different mass. This
generalization inherits the convexity and computational efficiency from the
Wasserstein-1 metric, and it includes several previous approaches from the
literature as special cases. For various specific instances of the generalized
Wasserstein-1 metric we furthermore demonstrate their usefulness in
applications by numerical experiments.
Exact and approximate inference in graphical models: variable elimination and beyond
Authors: Nathalie Peyrard, Marie-Josée Cros, Simon de Givry, Alain Franc, Stéphane Robin, Régis Sabbadin, Thomas Schiex, Matthieu Vignes
Date: 2015-06-29 00:00:00

Probabilistic graphical models offer a powerful framework to account for the
dependence structure between variables, which is represented as a graph.
However, the dependence between variables may render inference tasks
intractable. In this paper we review techniques exploiting the graph structure
for exact inference, borrowed from optimisation and computer science. They are
built on the principle of variable elimination whose complexity is dictated in
an intricate way by the order in which variables are eliminated. The so-called
treewidth of the graph characterises this algorithmic complexity: low-treewidth
graphs can be processed efficiently. The first message that we illustrate is
therefore the idea that for inference in graphical model, the number of
variables is not the limiting factor, and it is worth checking for the
treewidth before turning to approximate methods. We show how algorithms
providing an upper bound of the treewidth can be exploited to derive a 'good'
elimination order enabling to perform exact inference. The second message is
that when the treewidth is too large, algorithms for approximate inference
linked to the principle of variable elimination, such as loopy belief
propagation and variational approaches, can lead to accurate results while
being much less time consuming than Monte-Carlo approaches. We illustrate the
techniques reviewed in this article on benchmarks of inference problems in
genetic linkage analysis and computer vision, as well as on hidden variables
restoration in coupled Hidden Markov Models.
Deep Class-Wise Hashing: Semantics-Preserving Hashing via Class-wise Loss
Authors: Xuefei Zhe, Shifeng Chen, Hong Yan
Date: 2018-03-12 00:00:00
Tasks: Deep Hashing, Image Retrieval, Metric Learning, Retrieval, Triplet

Deep supervised hashing has emerged as an influential solution to large-scale
semantic image retrieval problems in computer vision. In the light of recent
progress, convolutional neural network based hashing methods typically seek
pair-wise or triplet labels to conduct the similarity preserving learning.
However, complex semantic concepts of visual contents are hard to capture by
similar/dissimilar labels, which limits the retrieval performance. Generally,
pair-wise or triplet losses not only suffer from expensive training costs but
also lack in extracting sufficient semantic information. In this regard, we
propose a novel deep supervised hashing model to learn more compact class-level
similarity preserving binary codes. Our deep learning based model is motivated
by deep metric learning that directly takes semantic labels as supervised
information in training and generates corresponding discriminant hashing code.
Specifically, a novel cubic constraint loss function based on Gaussian
distribution is proposed, which preserves semantic variations while penalizes
the overlap part of different classes in the embedding space. To address the
discrete optimization problem introduced by binary codes, a two-step
optimization strategy is proposed to provide efficient training and avoid the
problem of gradient vanishing. Extensive experiments on four large-scale
benchmark databases show that our model can achieve the state-of-the-art
retrieval performance. Moreover, when training samples are limited, our method
surpasses other supervised deep hashing methods with non-negligible margins.
Innovative Texture Database Collecting Approach and Feature Extraction Method based on Combination of Gray Tone Difference Matrixes, Local Binary Patterns,and K-means Clustering
Authors: Shervan Fekri-Ershad
Date: 2018-03-12 00:00:00
Tasks: Classification, Clustering, General Classification, Object Tracking, Texture Classification

Texture analysis and classification are some of the problems which have been
paid much attention by image processing scientists since late 80s. If texture
analysis is done accurately, it can be used in many cases such as object
tracking, visual pattern recognition, and face recognition.Since now, so many
methods are offered to solve this problem. Against their technical differences,
all of them used same popular databases to evaluate their performance such
asBrodatz or Outex, which may be made their performance biased on these
databases. In this paper, an approach is proposed to collect more efficient
databases of texture images. The proposed approach is included two stages. The
first one is developing feature representation based on gray tone difference
matrixes and local binary patterns features and the next one is consisted an
innovative algorithm which is based on K-means clustering to collect images
based on evaluated features. In order to evaluate the performance of the
proposed approach, a texture database is collected and fisher rate is computed
for collected one and well known databases. Also, texture classification is
evaluated based on offered feature extraction and the accuracy is compared by
some state of the art texture classification methods.
Estimating prediction error for complex samples
Authors: Andrew Holbrook, Thomas Lumley, Daniel Gillen
Date: 2017-11-13 00:00:00
Tasks: Prediction, Survey, Survey Sampling, Thompson Sampling

With a growing interest in using non-representative samples to train prediction models for numerous outcomes it is necessary to account for the sampling design that gives rise to the data in order to assess the generalized predictive utility of a proposed prediction rule. After learning a prediction rule based on a non-uniform sample, it is of interest to estimate the rule's error rate when applied to unobserved members of the population. Efron (1986) proposed a general class of covariance penalty inflated prediction error estimators that assume the available training data are representative of the target population for which the prediction rule is to be applied. We extend Efron's estimator to the complex sample context by incorporating Horvitz-Thompson sampling weights and show that it is consistent for the true generalization error rate when applied to the underlying superpopulation. The resulting Horvitz-Thompson-Efron (HTE) estimator is equivalent to dAIC, a recent extension of AIC to survey sampling data, but is more widely applicable. The proposed methodology is assessed with simulations and is applied to models predicting renal function obtained from the large-scale NHANES survey.
A Deep Learning Based Behavioral Approach to Indoor Autonomous Navigation
Authors: Gabriel Sepulveda, Juan Carlos Niebles, Alvaro Soto
Date: 2018-03-12 00:00:00
Tasks: Autonomous Navigation, Deep Learning

We present a semantically rich graph representation for indoor robotic
navigation. Our graph representation encodes: semantic locations such as
offices or corridors as nodes, and navigational behaviors such as enter office
or cross a corridor as edges. In particular, our navigational behaviors operate
directly from visual inputs to produce motor controls and are implemented with
deep learning architectures. This enables the robot to avoid explicit
computation of its precise location or the geometry of the environment, and
enables navigation at a higher level of semantic abstraction. We evaluate the
effectiveness of our representation by simulating navigation tasks in a large
number of virtual environments. Our results show that using a simple sets of
perceptual and navigational behaviors, the proposed approach can successfully
guide the way of the robot as it completes navigational missions such as going
to a specific office. Furthermore, our implementation shows to be effective to
control the selection and switching of behaviors.
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
Authors: Zhichao Yin, Jianping Shi
Date: 2018-03-06 00:00:00
Proceeding: CVPR 2018 6
Tasks: Camera Pose Estimation, Image Reconstruction, Motion Estimation, Optical Flow Estimation, Pose Estimation

We propose GeoNet, a jointly unsupervised learning framework for monocular
depth, optical flow and ego-motion estimation from videos. The three components
are coupled by the nature of 3D scene geometry, jointly learned by our
framework in an end-to-end manner. Specifically, geometric relationships are
extracted over the predictions of individual modules and then combined as an
image reconstruction loss, reasoning about static and dynamic scene parts
separately. Furthermore, we propose an adaptive geometric consistency loss to
increase robustness towards outliers and non-Lambertian regions, which resolves
occlusions and texture ambiguities effectively. Experimentation on the KITTI
driving dataset reveals that our scheme achieves state-of-the-art results in
all of the three tasks, performing better than previously unsupervised methods
and comparably with supervised ones.
Link prediction for egocentrically sampled networks
Authors: Yun-Jhong Wu, Elizaveta Levina, Ji Zhu
Date: 2018-03-12 00:00:00
Tasks: Graphon Estimation, Link Prediction, Prediction

Link prediction in networks is typically accomplished by estimating or
ranking the probabilities of edges for all pairs of nodes. In practice,
especially for social networks, the data are often collected by egocentric
sampling, which means selecting a subset of nodes and recording all of their
edges. This sampling mechanism requires different prediction tools than the
typical assumption of links missing at random. We propose a new computationally
efficient link prediction algorithm for egocentrically sampled networks, which
estimates the underlying probability matrix by estimating its row space. For
networks created by sampling rows, our method outperforms many popular link
prediction and graphon estimation techniques.
The biglasso Package: A Memory- and Computation-Efficient Solver for Lasso Model Fitting with Big Data in R
Authors: Yaohui Zeng, Patrick Breheny
Date: 2017-01-20 00:00:00
Tasks: Benchmarking

Penalized regression models such as the lasso have been extensively applied
to analyzing high-dimensional data sets. However, due to memory limitations,
existing R packages like glmnet and ncvreg are not capable of fitting
lasso-type models for ultrahigh-dimensional, multi-gigabyte data sets that are
increasingly seen in many areas such as genetics, genomics, biomedical imaging,
and high-frequency finance. In this research, we implement an R package called
biglasso that tackles this challenge. biglasso utilizes memory-mapped files to
store the massive data on the disk, only reading data into memory when
necessary during model fitting, and is thus able to handle out-of-core
computation seamlessly. Moreover, it's equipped with newly proposed, more
efficient feature screening rules, which substantially accelerate the
computation. Benchmarking experiments show that our biglasso package, as
compared to existing popular ones like glmnet, is much more memory- and
computation-efficient. We further analyze a 31 GB real data set on a laptop
with only 16 GB RAM to demonstrate the out-of-core computation capability of
biglasso in analyzing massive data sets that cannot be accommodated by existing
R packages.
Learning Local Distortion Visibility From Image Quality Data-sets
Authors: Navaneeth Kamballur Kottayil, Giuseppe Valenzise, Frederic Dufaux, Irene Cheng
Date: 2018-03-11 00:00:00
Tasks: Local Distortion

Accurate prediction of local distortion visibility thresholds is critical in
many image and video processing applications. Existing methods require an
accurate modeling of the human visual system, and are derived through
pshycophysical experiments with simple, artificial stimuli. These approaches,
however, are difficult to generalize to natural images with complex types of
distortion. In this paper, we explore a different perspective, and we
investigate whether it is possible to learn local distortion visibility from
image quality scores. We propose a convolutional neural network based
optimization framework to infer local detection thresholds in a distorted
image. Our model is trained on multiple quality datasets, and the results are
correlated with empirical visibility thresholds collected on complex stimuli in
a recent study. Our results are comparable to state-of-the-art mathematical
models that were trained on phsycovisual data directly. This suggests that it
is possible to predict psychophysical phenomena from visibility information
embedded in image quality scores.
Multiple Instance Choquet Integral Classifier Fusion and Regression for Remote Sensing Applications
Authors: Xiaoxiao Du, Alina Zare
Date: 2018-03-11 00:00:00
Tasks: Crop Yield Prediction, General Classification, Multiple Instance Learning, regression

In classifier (or regression) fusion the aim is to combine the outputs of
several algorithms to boost overall performance. Standard supervised fusion
algorithms often require accurate and precise training labels. However,
accurate labels may be difficult to obtain in many remote sensing applications.
This paper proposes novel classification and regression fusion models that can
be trained given ambiguosly and imprecisely labeled training data in which
training labels are associated with sets of data points (i.e., "bags") instead
of individual data points (i.e., "instances") following a multiple instance
learning framework. Experiments were conducted based on the proposed algorithms
on both synthetic data and applications such as target detection and crop yield
prediction given remote sensing data. The proposed algorithms show effective
classification and regression performance.
Interpreting Deep Classifier by Visual Distillation of Dark Knowledge
Authors: Kai Xu, Dae Hoon Park, Chang Yi, Charles Sutton
Date: 2018-03-11 00:00:00
Tasks: Dimensionality Reduction, Model Compression

Interpreting black box classifiers, such as deep networks, allows an analyst
to validate a classifier before it is deployed in a high-stakes setting. A
natural idea is to visualize the deep network's representations, so as to "see
what the network sees". In this paper, we demonstrate that standard dimension
reduction methods in this setting can yield uninformative or even misleading
visualizations. Instead, we present DarkSight, which visually summarizes the
predictions of a classifier in a way inspired by notion of dark knowledge.
DarkSight embeds the data points into a low-dimensional space such that it is
easy to compress the deep classifier into a simpler one, essentially combining
model compression and dimension reduction. We compare DarkSight against t-SNE
both qualitatively and quantitatively, demonstrating that DarkSight
visualizations are more informative. Our method additionally yields a new
confidence measure based on dark knowledge by quantifying how unusual a given
vector of predictions is.
Combinatorial Multi-Objective Multi-Armed Bandit Problem
Authors: Doruk Öner, Altuğ Karakurt, Atilla Eryilmaz, Cem Tekin
Date: 2018-03-11 00:00:00

In this paper, we introduce the COmbinatorial Multi-Objective Multi-Armed
Bandit (COMO-MAB) problem that captures the challenges of combinatorial and
multi-objective online learning simultaneously. In this setting, the goal of
the learner is to choose an action at each time, whose reward vector is a
linear combination of the reward vectors of the arms in the action, to learn
the set of super Pareto optimal actions, which includes the Pareto optimal
actions and actions that become Pareto optimal after adding an arbitrary small
positive number to their expected reward vectors. We define the Pareto regret
performance metric and propose a fair learning algorithm whose Pareto regret is
$O(N L^3 \log T)$, where $T$ is the time horizon, $N$ is the number of arms and
$L$ is the maximum number of arms in an action. We show that COMO-MAB has a
wide range of applications, including recommending bundles of items to users
and network routing, and focus on a resource-allocation application for
multi-user communication in the presence of multidimensional performance
metrics, where we show that our algorithm outperforms existing MAB algorithms.
Sales forecasting using WaveNet within the framework of the Kaggle competition
Authors: Glib Kechyn, Lucius Yu, Yangguang Zang, Svyatoslav Kechyn
Date: 2018-03-11 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

We took part in the Corporacion Favorita Grocery Sales Forecasting
competition hosted on Kaggle and achieved the 2nd place. In this abstract
paper, we present an overall analysis and solution to the underlying
machine-learning problem based on time series data, where major challenges are
identified and corresponding preliminary methods are proposed. Our approach is
based on the adaptation of dilated convolutional neural network for time series
forecasting. By applying this technique iteratively to batches of n examples, a
big amount of time series data can be eventually processed with a decent speed
and accuracy. We hope this paper could serve, to some extent, as a review and
guideline of the time series forecasting benchmark, inspiring further attempts
and researches.
Cascade context encoder for improved inpainting
Authors: Bartosz Zieliński, Łukasz Struski, Marek Śmieja, Jacek Tabor
Date: 2018-03-11 00:00:00

In this paper, we analyze if cascade usage of the context encoder with
increasing input can improve the results of the inpainting. For this purpose,
we train context encoder for 64x64 pixels images in a standard way and use its
resized output to fill in the missing input region of the 128x128 context
encoder, both in training and evaluation phase. As the result, the inpainting
is visibly more plausible. In order to thoroughly verify the results, we
introduce normalized squared-distortion, a measure for quantitative inpainting
evaluation, and we provide its mathematical explanation. This is the first
attempt to formalize the inpainting measure, which is based on the properties
of latent feature representation, instead of L2 reconstruction loss.
Separation of time scales and direct computation of weights in deep neural networks
Authors: Nima Dehmamy, Neda Rohani, Aggelos Katsaggelos
Date: 2017-03-14 00:00:00

Artificial intelligence is revolutionizing our lives at an ever increasing
pace. At the heart of this revolution is the recent advancements in deep neural
networks (DNN), learning to perform sophisticated, high-level tasks. However,
training DNNs requires massive amounts of data and is very computationally
intensive. Gaining analytical understanding of the solutions found by DNNs can
help us devise more efficient training algorithms, replacing the commonly used
mthod of stochastic gradient descent (SGD). We analyze the dynamics of SGD and
show that, indeed, direct computation of the solutions is possible in many
cases. We show that a high performing setup used in DNNs introduces a
separation of time-scales in the training dynamics, allowing SGD to train
layers from the lowest (closest to input) to the highest. We then show that for
each layer, the distribution of solutions found by SGD can be estimated using a
class-based principal component analysis (PCA) of the layer's input. This
finding allows us to forgo SGD entirely and directly derive the DNN parameters
using this class-based PCA, which can be well estimated using significantly
less data than SGD. We implement these results on image datasets MNIST, CIFAR10
and CIFAR100 and find that, in fact, layers derived using our class-based PCA
perform comparable or superior to neural networks of the same size and
architecture trained using SGD. We also confirm that the class-based PCA often
converges using a fraction of the data required for SGD. Thus, using our method
training time can be reduced both by requiring less training data than SGD, and
by eliminating layers in the costly backpropagation step of the training.
DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild
Authors: Riza Alp Guler, Yuxiang Zhou, George Trigeorgis, Epameinondas Antonakos, Patrick Snape, Stefanos Zafeiriou, Iasonas Kokkinos
Date: 2018-03-05 00:00:00
Proceeding: CVPR 2017
Tasks: Face Alignment, Pose Estimation, regression, Semantic Segmentation

In this work we use deep learning to establish dense correspondences between
a 3D object model and an image "in the wild". We introduce "DenseReg", a
fully-convolutional neural network (F-CNN) that densely regresses at every
foreground pixel a pair of U-V template coordinates in a single feedforward
pass. To train DenseReg we construct a supervision signal by combining 3D
deformable model fitting and 2D landmark annotations. We define the regression
task in terms of the intrinsic, U-V coordinates of a 3D deformable model that
is brought into correspondence with image instances at training time. A host of
other object-related tasks (e.g. part segmentation, landmark localization) are
shown to be by-products of this task, and to largely improve thanks to its
introduction. We obtain highly-accurate regression results by combining ideas
from semantic segmentation with regression networks, yielding a 'quantized
regression' architecture that first obtains a quantized estimate of position
through classification, and refines it through regression of the residual. We
show that such networks can boost the performance of existing state-of-the-art
systems for pose estimation. Firstly, we show that our system can serve as an
initialization for Statistical Deformable Models, as well as an element of
cascaded architectures that jointly localize landmarks and estimate dense
correspondences. We also show that the obtained dense correspondence can act as
a source of 'privileged information' that complements and extends the pure
landmark-level annotations, accelerating and improving the training of pose
estimation networks. We report state-of-the-art performance on the challenging
300W benchmark for facial landmark localization and on the MPII and LSP
datasets for human pose estimation.