pwc-archive/papers-with-abstracts huggingface dataset:
Differentially Private Bayesian Inference for Exponential Families
Authors: Garrett Bernstein, Daniel Sheldon
Date: 2018-09-06 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Bayesian Inference

The study of private inference has been sparked by growing concern regarding
the analysis of data when it stems from sensitive sources. We present the first
method for private Bayesian inference in exponential families that properly
accounts for noise introduced by the privacy mechanism. It is efficient because
it works only with sufficient statistics and not individual data. Unlike other
methods, it gives properly calibrated posterior beliefs in the non-asymptotic
data regime.
Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings
Authors: Mohsan Alvi, Andrew Zisserman, Christoffer Nellaker
Date: 2018-09-06 00:00:00
Tasks: Attribute, Classification, Facial Attribute Classification, General Classification, image-classification, Image Classification

Neural networks achieve the state-of-the-art in image classification tasks.
However, they can encode spurious variations or biases that may be present in
the training data. For example, training an age predictor on a dataset that is
not balanced for gender can lead to gender biased predicitons (e.g. wrongly
predicting that males are older if only elderly males are in the training set).
We present two distinct contributions: 1) An algorithm that can remove multiple
sources of variation from the feature representation of a network. We
demonstrate that this algorithm can be used to remove biases from the feature
representation, and thereby improve classification accuracies, when training
networks on extremely biased datasets. 2) An ancestral origin database of
14,000 images of individuals from East Asia, the Indian subcontinent,
sub-Saharan Africa, and Western Europe. We demonstrate on this dataset, for a
number of facial attribute classification tasks, that we are able to remove
racial biases from the network feature representation.
Deep Learning for Generic Object Detection: A Survey
Authors: Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, Matti Pietikäinen
Date: 2018-09-06 00:00:00
Tasks: Deep Learning, Object, object-detection, Object Detection, Object Proposal Generation, Survey

Object detection, one of the most fundamental and challenging problems in computer vision, seeks to locate object instances from a large number of predefined categories in natural images. Deep learning techniques have emerged as a powerful strategy for learning feature representations directly from data and have led to remarkable breakthroughs in the field of generic object detection. Given this period of rapid evolution, the goal of this paper is to provide a comprehensive survey of the recent achievements in this field brought about by deep learning techniques. More than 300 research contributions are included in this survey, covering many aspects of generic object detection: detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics. We finish the survey by identifying promising directions for future research.
Escaping Saddle Points in Constrained Optimization
Authors: Aryan Mokhtari, Asuman Ozdaglar, Ali Jadbabaie
Date: 2018-09-06 00:00:00
Proceeding: NeurIPS 2018 12

In this paper, we study the problem of escaping from saddle points in smooth
nonconvex optimization problems subject to a convex set $\mathcal{C}$. We
propose a generic framework that yields convergence to a second-order
stationary point of the problem, if the convex set $\mathcal{C}$ is simple for
a quadratic objective function. Specifically, our results hold if one can find
a $\rho$-approximate solution of a quadratic program subject to $\mathcal{C}$
in polynomial time, where $\rho<1$ is a positive constant that depends on the
structure of the set $\mathcal{C}$. Under this condition, we show that the
sequence of iterates generated by the proposed framework reaches an
$(\epsilon,\gamma)$-second order stationary point (SOSP) in at most
$\mathcal{O}(\max\{\epsilon^{-2},\rho^{-3}\gamma^{-3}\})$ iterations. We
further characterize the overall complexity of reaching an SOSP when the convex
set $\mathcal{C}$ can be written as a set of quadratic constraints and the
objective function Hessian has a specific structure over the convex set
$\mathcal{C}$. Finally, we extend our results to the stochastic setting and
characterize the number of stochastic gradient and Hessian evaluations to reach
an $(\epsilon,\gamma)$-SOSP.
Scalable Learning in Reproducing Kernel Krein Spaces
Authors: Dino Oglic, Thomas Gärtner
Date: 2018-09-06 00:00:00
Tasks: Time Series, Time Series Analysis

We provide the first mathematically complete derivation of the Nystr\"om method for low-rank approximation of indefinite kernels and propose an efficient method for finding an approximate eigendecomposition of such kernel matrices. Building on this result, we devise highly scalable methods for learning in reproducing kernel Kre\u{\i}n spaces. The devised approaches provide a principled and theoretically well-founded means to tackle large scale learning problems with indefinite kernels. The main motivation for our work comes from problems with structured representations (e.g., graphs, strings, time-series), where it is relatively easy to devise a pairwise (dis)similarity function based on intuition and/or knowledge of domain experts. Such functions are typically not positive definite and it is often well beyond the expertise of practitioners to verify this condition. The effectiveness of the devised approaches is evaluated empirically using indefinite kernels defined on structured and vectorial data representations.
Object Hallucination in Image Captioning
Authors: Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, Kate Saenko
Date: 2018-09-06 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Hallucination, Image Captioning, Object, Object Hallucination, Sentence

Despite continuously improving performance, contemporary image captioning
models are prone to "hallucinating" objects that are not actually in a scene.
One problem is that standard metrics only measure similarity to ground truth
captions and may not fully capture image relevance. In this work, we propose a
new image relevance metric to evaluate current models with veridical visual
labels and assess their rate of object hallucination. We analyze how captioning
model architectures and learning objectives contribute to object hallucination,
explore when hallucination is likely due to image misclassification or language
priors, and assess how well current sentence metrics capture object
hallucination. We investigate these questions on the standard image captioning
benchmark, MSCOCO, using a diverse set of models. Our analysis yields several
interesting findings, including that models which score best on standard
sentence metrics do not always have lower hallucination and that models which
hallucinate more tend to make errors driven by language priors.
From FATS to feets: Further improvements to an astronomical feature extraction tool based on machine learning
Authors: J. B. Cabral, B. Sánchez, F. Ramos, S. Gurovich, P. Granitto, J. Vanderplas
Date: 2018-09-06 00:00:00
Tasks: Astronomy, BIG-bench Machine Learning, General Classification, Time Series, Time Series Analysis

Machine learning algorithms are highly useful for the classification of time
series data in astronomy in this era of peta-scale public survey data releases.
These methods can facilitate the discovery of new unknown events in most
astrophysical areas, as well as improving the analysis of samples of known
phenomena. Machine learning algorithms use features extracted from collected
data as input predictive variables. A public tool called Feature Analysis for
Time Series (FATS) has proved an excellent workhorse for feature extraction,
particularly light curve classification for variable objects. In this study, we
present a major improvement to FATS, which corrects inconvenient design
choices, minor details, and documentation for the re-engineering process. This
improvement comprises a new Python package called "feets", which is important
for future code-refactoring for astronomical software tools.
Variational Bayesian Inference for Robust Streaming Tensor Factorization and Completion
Authors: Cole Hawkins, Zheng Zhang
Date: 2018-09-06 00:00:00
Tasks: Bayesian Inference, Recommendation Systems

Streaming tensor factorization is a powerful tool for processing high-volume
and multi-way temporal data in Internet networks, recommender systems and
image/video data analysis. Existing streaming tensor factorization algorithms
rely on least-squares data fitting and they do not possess a mechanism for
tensor rank determination. This leaves them susceptible to outliers and
vulnerable to over-fitting. This paper presents a Bayesian robust streaming
tensor factorization model to identify sparse outliers, automatically determine
the underlying tensor rank and accurately fit low-rank structure. We implement
our model in Matlab and compare it with existing algorithms on tensor datasets
generated from dynamic MRI and Internet traffic.
Upcycle Your OCR: Reusing OCRs for Post-OCR Text Correction in Romanised Sanskrit
Authors: Amrith Krishna, Bodhisattwa Prasad Majumder, Rajesh Shreedhar Bhat, Pawan Goyal
Date: 2018-09-06 00:00:00
Proceeding: CONLL 2018 10
Tasks: Optical Character Recognition (OCR)

We propose a post-OCR text correction approach for digitising texts in
Romanised Sanskrit. Owing to the lack of resources our approach uses OCR models
trained for other languages written in Roman. Currently, there exists no
dataset available for Romanised Sanskrit OCR. So, we bootstrap a dataset of 430
images, scanned in two different settings and their corresponding ground truth.
For training, we synthetically generate training images for both the settings.
We find that the use of copying mechanism (Gu et al., 2016) yields a percentage
increase of 7.69 in Character Recognition Rate (CRR) than the current state of
the art model in solving monotone sequence-to-sequence tasks (Schnober et al.,
2016). We find that our system is robust in combating OCR-prone errors, as it
obtains a CRR of 87.01% from an OCR output with CRR of 35.76% for one of the
dataset settings. A human judgment survey performed on the models shows that
our proposed model results in predictions which are faster to comprehend and
faster to improve for a human than the other systems.
GANs beyond divergence minimization
Authors: Alexia Jolicoeur-Martineau
Date: 2018-09-06 00:00:00

Generative adversarial networks (GANs) can be interpreted as an adversarial
game between two players, a discriminator D and a generator G, in which D
learns to classify real from fake data and G learns to generate realistic data
by "fooling" D into thinking that fake data is actually real data. Currently, a
dominating view is that G actually learns by minimizing a divergence given that
the general objective function is a divergence when D is optimal. However, this
view has been challenged due to inconsistencies between theory and practice. In
this paper, we discuss of the properties associated with most loss functions
for G (e.g., saturating/non-saturating f-GAN, LSGAN, WGAN, etc.). We show that
these loss functions are not divergences and do not have the same equilibrium
as expected of divergences. This suggests that G does not need to minimize the
same objective function as D maximize, nor maximize the objective of D after
swapping real data with fake data (non-saturating GAN) but can instead use a
wide range of possible loss functions to learn to generate realistic data. We
define GANs through two separate and independent D maximization and G
minimization steps. We generalize the generator step to four new classes of
loss functions, most of which are actual divergences (while traditional G loss
functions are not). We test a wide variety of loss functions from these four
classes on a synthetic dataset and on CIFAR-10. We observe that most loss
functions converge well and provide comparable data generation quality to
non-saturating GAN, LSGAN, and WGAN-GP generator loss functions, whether we use
divergences or non-divergences. These results suggest that GANs do not conform
well to the divergence minimization theory and form a much broader range of
models than previously assumed.
Balanced multi-shot EPI for accelerated Cartesian MRF: An alternative to spiral MRF
Authors: Arnold Julian Vinoj Benjamin, Pedro A. Gómez, Mohammad Golbabaee, Tim Sprenger, Marion I. Menzel, Mike E. Davies, Ian Marshall
Date: 2018-09-06 00:00:00

The main purpose of this study is to show that a highly accelerated Cartesian
MRF scheme using a multi-shot EPI readout (i.e. multi-shot EPI-MRF) can produce
good quality multi-parametric maps such as T1, T2 and proton density (PD) in a
sufficiently short scan duration that is similar to conventional MRF. This
multi-shot approach allows considerable subsampling while traversing the entire
k-space trajectory, can yield better SNR, reduced blurring, less distortion and
can also be used to collect higher resolution data compared to existing
single-shot EPI-MRF implementations. The generated parametric maps are compared
to an accelerated spiral MRF implementation with the same acquisition
parameters to evaluate the performance of this method. Additionally, an
iterative reconstruction algorithm is applied to improve the accuracy of
parametric map estimations and the fast convergence of EPI-MRF is also
demonstrated.
Stochastically Controlled Stochastic Gradient for the Convex and Non-convex Composition problem
Authors: Liu Liu, Ji Liu, Cho-Jui Hsieh, DaCheng Tao
Date: 2018-09-06 00:00:00

In this paper, we consider the convex and non-convex composition problem with
the structure $\frac{1}{n}\sum\nolimits_{i = 1}^n {{F_i}( {G( x )} )}$, where
$G( x )=\frac{1}{n}\sum\nolimits_{j = 1}^n {{G_j}( x )} $ is the inner
function, and $F_i(\cdot)$ is the outer function. We explore the variance
reduction based method to solve the composition optimization. Due to the fact
that when the number of inner function and outer function are large, it is not
reasonable to estimate them directly, thus we apply the stochastically
controlled stochastic gradient (SCSG) method to estimate the gradient of the
composition function and the value of the inner function. The query complexity
of our proposed method for the convex and non-convex problem is equal to or
better than the current method for the composition problem. Furthermore, we
also present the mini-batch version of the proposed method, which has the
improved the query complexity with related to the size of the mini-batch.
CoverBLIP: scalable iterative matched filtering for MR Fingerprint recovery
Authors: Mohammad Golbabaee, Zhouye Chen, Yves Wiaux, Mike E. Davies
Date: 2018-09-06 00:00:00

Current proposed solutions for the high dimensionality of the MRF
reconstruction problem rely on a linear compression step to reduce the matching
computations and boost the efficiency of fast but non-scalable searching
schemes such as the KD-trees. However such methodologies often introduce an
unfavourable compromise in the estimation accuracy when applied to nonlinear
data structures such as the manifold of Bloch responses with possible increased
dynamic complexity and growth in data population. To address this shortcoming
we propose an inexact iterative reconstruction method, dubbed as the Cover
BLoch response Iterative Projection (CoverBLIP). Iterative methods improve the
accuracy of their non-iterative counterparts and are additionally robust
against certain accelerated approximate updates, without compromising their
final accuracy. Leveraging on these results, we accelerate matched-filtering
using an ANNS algorithm based on Cover trees with a robustness feature against
the curse of dimensionality.
On the Importance of Visual Context for Data Augmentation in Scene Understanding
Authors: Nikita Dvornik, Julien Mairal, Cordelia Schmid
Date: 2018-09-06 00:00:00
Tasks: Data Augmentation, Instance Segmentation, Object, object-detection, Object Detection, Scene Understanding, Segmentation, Semantic Segmentation, Weakly-supervised Learning

Performing data augmentation for learning deep neural networks is known to be important for training visual recognition systems. By artificially increasing the number of training examples, it helps reducing overfitting and improves generalization. While simple image transformations can already improve predictive performance in most vision tasks, larger gains can be obtained by leveraging task-specific prior knowledge. In this work, we consider object detection, semantic and instance segmentation and augment the training images by blending objects in existing scenes, using instance segmentation annotations. We observe that randomly pasting objects on images hurts the performance, unless the object is placed in the right context. To resolve this issue, we propose an explicit context model by using a convolutional neural network, which predicts whether an image region is suitable for placing a given object or not. In our experiments, we show that our approach is able to improve object detection, semantic and instance segmentation on the PASCAL VOC12 and COCO datasets, with significant gains in a limited annotation scenario, i.e. when only one category is annotated. We also show that the method is not limited to datasets that come with expensive pixel-wise instance annotations and can be used when only bounding boxes are available, by employing weakly-supervised learning for instance masks approximation.
Five lessons from building a deep neural network recommender
Authors: Simen Eide, Audun M. Øygard, Ning Zhou
Date: 2018-09-06 00:00:00
Tasks: Recommendation Systems, Transfer Learning

Recommendation algorithms are widely adopted in marketplaces to help users
find the items they are looking for. The sparsity of the items by user matrix
and the cold-start issue in marketplaces pose challenges for the off-the-shelf
matrix factorization based recommender systems. To understand user intent and
tailor recommendations to their needs, we use deep learning to explore various
heterogeneous data available in marketplaces. This paper summarizes five
lessons we learned from experimenting with state-of-the-art deep learning
recommenders at the leading Norwegian marketplace FINN.no. We design a hybrid
recommender system that takes the user-generated contents of a marketplace
(including text, images and meta attributes) and combines them with user
behavior data such as page views and messages to provide recommendations for
marketplace items. Among various tactics we experimented with, the following
five show the best impact: staged training instead of end-to-end training,
leveraging rich user behaviors beyond page views, using user behaviors as noisy
labels to train embeddings, using transfer learning to solve the unbalanced
data problem, and using attention mechanisms in the hybrid model. This system
is currently running with around 20% click-through-rate in production at
FINN.no and serves over one million visitors everyday.
Deep neural network marketplace recommenders in online experiments
Authors: Simen Eide, Ning Zhou
Date: 2018-09-06 00:00:00
Tasks: Re-Ranking

Recommendations are broadly used in marketplaces to match users with items
relevant to their interests and needs. To understand user intent and tailor
recommendations to their needs, we use deep learning to explore various
heterogeneous data available in marketplaces. This paper focuses on the
challenge of measuring recommender performance and summarizes the online
experiment results with several promising types of deep neural network
recommenders - hybrid item representation models combining features from user
engagement and content, sequence-based models, and multi-armed bandit models
that optimize user engagement by re-ranking proposals from multiple submodels.
The recommenders are currently running in production at the leading Norwegian
marketplace FINN.no and serves over one million visitors everyday.
A note on concentration inequality for vector-valued martingales with weak exponential-type tails
Authors: Chris Junchi Li
Date: 2018-09-06 00:00:00
Tasks: Dimensionality Reduction, LEMMA

We present novel martingale concentration inequalities for martingale differences with finite Orlicz-$\psi_\alpha$ norms. Such martingale differences with weak exponential-type tails scatters in many statistical applications and can be heavier than sub-exponential distributions. In the case of one dimension, we prove in general that for a sequence of scalar-valued supermartingale difference, the tail bound depends solely on the sum of squared Orlicz-$\psi_\alpha$ norms instead of the maximal Orlicz-$\psi_\alpha$ norm, generalizing the results of Lesigne & Voln\'y (2001) and Fan et al. (2012). In the multidimensional case, using a dimension reduction lemma proposed by Kallenberg & Sztencel (1991) we show that essentially the same concentration tail bound holds for vector-valued martingale difference sequences.
Sample Efficient Stochastic Gradient Iterative Hard Thresholding Method for Stochastic Sparse Linear Regression with Limited Attribute Observation
Authors: Tomoya Murata, Taiji Suzuki
Date: 2018-09-05 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Attribute

We develop new stochastic gradient methods for efficiently solving sparse
linear regression in a partial attribute observation setting, where learners
are only allowed to observe a fixed number of actively chosen attributes per
example at training and prediction times. It is shown that the methods achieve
essentially a sample complexity of $O(1/\varepsilon)$ to attain an error of
$\varepsilon$ under a variant of restricted eigenvalue condition, and the rate
has better dependency on the problem dimension than existing methods.
Particularly, if the smallest magnitude of the non-zero components of the
optimal solution is not too small, the rate of our proposed {\it Hybrid}
algorithm can be boosted to near the minimax optimal sample complexity of {\it
full information} algorithms. The core ideas are (i) efficient construction of
an unbiased gradient estimator by the iterative usage of the hard thresholding
operator for configuring an exploration algorithm; and (ii) an adaptive
combination of the exploration and an exploitation algorithms for quickly
identifying the support of the optimum and efficiently searching the optimal
parameter in its support. Experimental results are presented to validate our
theoretical findings and the superiority of our proposed methods.
Deep Relevance Ranking Using Enhanced Document-Query Interactions
Authors: Ryan McDonald, Georgios-Ioannis Brokos, Ion Androutsopoulos
Date: 2018-09-05 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Ad-Hoc Information Retrieval, Question Answering

We explore several new models for document relevance ranking, building upon
the Deep Relevance Matching Model (DRMM) of Guo et al. (2016). Unlike DRMM,
which uses context-insensitive encodings of terms and query-document term
interactions, we inject rich context-sensitive encodings throughout our models,
inspired by PACRR's (Hui et al., 2017) convolutional n-gram matching features,
but extended in several ways including multiple views of query and document
inputs. We test our models on datasets from the BIOASQ question answering
challenge (Tsatsaronis et al., 2015) and TREC ROBUST 2004 (Voorhees, 2005),
showing they outperform BM25-based baselines, DRMM, and PACRR.