pwc-archive/papers-with-abstracts huggingface dataset:
Collaborative Annotation of Semantic Objects in Images with Multi-granularity Supervisions
Authors: Lishi Zhang, Chenghan Fu, Jia Li
Date: 2018-06-27 00:00:00
Tasks: Object, Superpixels, TAG

Per-pixel masks of semantic objects are very useful in many applications,
which, however, are tedious to be annotated. In this paper, we propose a
human-agent collaborative annotation approach that can efficiently generate
per-pixel masks of semantic objects in tagged images with multi-granularity
supervisions. Given a set of tagged image, a computer agent is first
dynamically generated to roughly localize the semantic objects described by the
tag. The agent first extracts massive object proposals from an image and then
infer the tag-related ones under the weak and strong supervisions from
linguistically and visually similar images and previously annotated object
masks. By representing such supervisions by over-complete dictionaries, the
tag-related object proposals can pop-out according to their sparse coding
length, which are then converted to superpixels with binary labels. After that,
human annotators participate in the annotation process by flipping labels and
dividing superpixels with mouse clicks, which are used as click supervisions
that teach the agent to recover false positives/negatives in processing images
with the same tags. Experimental results show that our approach can facilitate
the annotation process and generate object masks that are highly consistent
with those generated by the LabelMe toolbox.
Learning a Saliency Evaluation Metric Using Crowdsourced Perceptual Judgments
Authors: Changqun Xia, Jia Li, Jinming Su, Ali Borji
Date: 2018-06-27 00:00:00
Tasks: Benchmarking

In the area of human fixation prediction, dozens of computational saliency
models are proposed to reveal certain saliency characteristics under different
assumptions and definitions. As a result, saliency model benchmarking often
requires several evaluation metrics to simultaneously assess saliency models
from multiple perspectives. However, most computational metrics are not
designed to directly measure the perceptual similarity of saliency maps so that
the evaluation results may be sometimes inconsistent with the subjective
impression. To address this problem, this paper first conducts extensive
subjective tests to find out how the visual similarities between saliency maps
are perceived by humans. Based on the crowdsourced data collected in these
tests, we conclude several key factors in assessing saliency maps and quantize
the performance of existing metrics. Inspired by these factors, we propose to
learn a saliency evaluation metric based on a two-stream convolutional neural
network using crowdsourced perceptual judgements. Specifically, the relative
saliency score of each pair from the crowdsourced data is utilized to
regularize the network during the training process. By capturing the key
factors shared by various subjects in comparing saliency maps, the learned
metric better aligns with human perception of saliency maps, making it a good
complement to the existing metrics. Experimental results validate that the
learned metric can be generalized to the comparisons of saliency maps from new
images, new datasets, new models and synthetic data. Due to the effectiveness
of the learned metric, it also can be used to facilitate the development of new
models for fixation prediction.
Phase transition in the knapsack problem
Authors: Nitin Yadav, Carsten Murawski, Sebastian Sardina, Peter Bossaerts
Date: 2018-06-26 00:00:00

We examine the phase transition phenomenon for the Knapsack problem from both
a computational and a human perspective. We first provide, via an empirical and
a theoretical analysis, a characterization of the phenomenon in terms of two
instance properties; normalised capacity and normalised profit. Then, we show
evidence that average time spent by human decision makers in solving an
instance peaks near the phase transition. Given the ubiquity of the Knapsack
problem in every-day life, a better understanding of its structure can improve
our understanding not only of computational techniques but also of human
behavior, including the use and development of heuristics and occurrence of
biases.
Scalable Gaussian Process Inference with Finite-data Mean and Variance Guarantees
Authors: Jonathan H. Huggins, Trevor Campbell, Mikołaj Kasprzak, Tamara Broderick
Date: 2018-06-26 00:00:00
Tasks: Gaussian Processes, regression, Variational Inference

Gaussian processes (GPs) offer a flexible class of priors for nonparametric
Bayesian regression, but popular GP posterior inference methods are typically
prohibitively slow or lack desirable finite-data guarantees on quality. We
develop an approach to scalable approximate GP regression with finite-data
guarantees on the accuracy of pointwise posterior mean and variance estimates.
Our main contribution is a novel objective for approximate inference in the
nonparametric setting: the preconditioned Fisher (pF) divergence. We show that
unlike the Kullback--Leibler divergence (used in variational inference), the pF
divergence bounds the 2-Wasserstein distance, which in turn provides tight
bounds the pointwise difference of the mean and variance functions. We
demonstrate that, for sparse GP likelihood approximations, we can minimize the
pF divergence efficiently. Our experiments show that optimizing the pF
divergence has the same computational requirements as variational sparse GPs
while providing comparable empirical performance--in addition to our novel
finite-data quality guarantees.
Guided evolutionary strategies: Augmenting random search with surrogate gradients
Authors: Niru Maheswaranathan, Luke Metz, George Tucker, Dami Choi, Jascha Sohl-Dickstein
Date: 2018-06-26 00:00:00
Proceeding: ICLR 2019 5
Tasks: Meta-Learning, Reinforcement Learning

Many applications in machine learning require optimizing a function whose true gradient is unknown, but where surrogate gradient information (directions that may be correlated with, but not necessarily identical to, the true gradient) is available instead. This arises when an approximate gradient is easier to compute than the full gradient (e.g. in meta-learning or unrolled optimization), or when a true gradient is intractable and is replaced with a surrogate (e.g. in certain reinforcement learning applications, or when using synthetic gradients). We propose Guided Evolutionary Strategies, a method for optimally using surrogate gradient directions along with random search. We define a search distribution for evolutionary strategies that is elongated along a guiding subspace spanned by the surrogate gradients. This allows us to estimate a descent direction which can then be passed to a first-order optimizer. We analytically and numerically characterize the tradeoffs that result from tuning how strongly the search distribution is stretched along the guiding subspace, and we use this to derive a setting of the hyperparameters that works well across problems. Finally, we apply our method to example problems, demonstrating an improvement over both standard evolutionary strategies and first-order methods (that directly follow the surrogate gradient). We provide a demo of Guided ES at https://github.com/brain-research/guided-evolutionary-strategies
Conditional Sparse $\ell_p$-norm Regression With Optimal Probability
Authors: John Hainline, Brendan Juba, Hai S. Le, David Woodruff
Date: 2018-06-26 00:00:00
Tasks: regression

We consider the following conditional linear regression problem: the task is
to identify both (i) a $k$-DNF condition $c$ and (ii) a linear rule $f$ such
that the probability of $c$ is (approximately) at least some given bound $\mu$,
and $f$ minimizes the $\ell_p$ loss of predicting the target $z$ in the
distribution of examples conditioned on $c$. Thus, the task is to identify a
portion of the distribution on which a linear rule can provide a good fit.
Algorithms for this task are useful in cases where simple, learnable rules only
accurately model portions of the distribution. The prior state-of-the-art for
such algorithms could only guarantee finding a condition of probability
$\Omega(\mu/n^k)$ when a condition of probability $\mu$ exists, and achieved an
$O(n^k)$-approximation to the target loss, where $n$ is the number of Boolean
attributes. Here, we give efficient algorithms for solving this task with a
condition $c$ that nearly matches the probability of the ideal condition, while
also improving the approximation to the target loss. We also give an algorithm
for finding a $k$-DNF reference class for prediction at a given query point,
that obtains a sparse regression fit that has loss within $O(n^k)$ of optimal
among all sparse regression parameters and sufficiently large $k$-DNF reference
classes containing the query point.
Contextual Language Model Adaptation for Conversational Agents
Authors: Anirudh Raju, Behnam Hedayatnia, Linda Liu, Ankur Gandhe, Chandra Khatri, Angeliki Metallinou, Anu Venkatesh, Ariya Rastrow
Date: 2018-06-26 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, model, speech-recognition, Speech Recognition

Statistical language models (LM) play a key role in Automatic Speech
Recognition (ASR) systems used by conversational agents. These ASR systems
should provide a high accuracy under a variety of speaking styles, domains,
vocabulary and argots. In this paper, we present a DNN-based method to adapt
the LM to each user-agent interaction based on generalized contextual
information, by predicting an optimal, context-dependent set of LM
interpolation weights. We show that this framework for contextual adaptation
provides accuracy improvements under different possible mixture LM partitions
that are relevant for both (1) Goal-oriented conversational agents where it's
natural to partition the data by the requested application and for (2) Non-goal
oriented conversational agents where the data can be partitioned using topic
labels that come from predictions of a topic classifier. We obtain a relative
WER improvement of 3% with a 1-pass decoding strategy and 6% in a 2-pass
decoding framework, over an unadapted model. We also show up to a 15% relative
improvement in recognizing named entities which is of significant value for
conversational ASR systems.
Deep Feature Factorization For Concept Discovery
Authors: Edo Collins, Radhakrishna Achanta, Sabine Süsstrunk
Date: 2018-06-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: Unsupervised Facial Landmark Detection, Unsupervised Human Pose Estimation, Unsupervised Keypoints

We propose Deep Feature Factorization (DFF), a method capable of localizing
similar semantic concepts within an image or a set of images. We use DFF to
gain insight into a deep convolutional neural network's learned features, where
we detect hierarchical cluster structures in feature space. This is visualized
as heat maps, which highlight semantically matching regions across a set of
images, revealing what the network `perceives' as similar. DFF can also be used
to perform co-segmentation and co-localization, and we report state-of-the-art
results on these tasks.
Neural Cross-Lingual Coreference Resolution and its Application to Entity Linking
Authors: Gourab Kundu, Avirup Sil, Radu Florian, Wael Hamza
Date: 2018-06-26 00:00:00
Proceeding: ACL 2018 7
Tasks: coreference-resolution, Coreference Resolution, Entity Linking

We propose an entity-centric neural cross-lingual coreference model that
builds on multi-lingual embeddings and language-independent features. We
perform both intrinsic and extrinsic evaluations of our model. In the intrinsic
evaluation, we show that our model, when trained on English and tested on
Chinese and Spanish, achieves competitive results to the models trained
directly on Chinese and Spanish respectively. In the extrinsic evaluation, we
show that our English model helps achieve superior entity linking accuracy on
Chinese and Spanish test sets than the top 2015 TAC system without using any
annotated data from Chinese or Spanish.
A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updates
Authors: Yossi Arjevani, Ohad Shamir, Nathan Srebro
Date: 2018-06-26 00:00:00
Tasks: Distributed Optimization

We provide tight finite-time convergence bounds for gradient descent and
stochastic gradient descent on quadratic functions, when the gradients are
delayed and reflect iterates from $\tau$ rounds ago. First, we show that
without stochastic noise, delays strongly affect the attainable optimization
error: In fact, the error can be as bad as non-delayed gradient descent ran on
only $1/\tau$ of the gradients. In sharp contrast, we quantify how stochastic
noise makes the effect of delays negligible, improving on previous work which
only showed this phenomenon asymptotically or for much smaller delays. Also, in
the context of distributed optimization, the results indicate that the
performance of gradient descent with delays is competitive with synchronous
approaches such as mini-batching. Our results are based on a novel technique
for analyzing convergence of optimization algorithms using generating
functions.
Dual SVM Training on a Budget
Authors: Sahar Qaadan, Merlin Schüler, Tobias Glasmachers
Date: 2018-06-26 00:00:00

We present a dual subspace ascent algorithm for support vector machine
training that respects a budget constraint limiting the number of support
vectors. Budget methods are effective for reducing the training time of kernel
SVM while retaining high accuracy. To date, budget training is available only
for primal (SGD-based) solvers. Dual subspace ascent methods like sequential
minimal optimization are attractive for their good adaptation to the problem
structure, their fast convergence rate, and their practical speed. By
incorporating a budget constraint into a dual algorithm, our method enjoys the
best of both worlds. We demonstrate considerable speed-ups over primal budget
training methods.
Unsupervised Learning by Competing Hidden Units
Authors: Dmitry Krotov, John Hopfield
Date: 2018-06-26 00:00:00

It is widely believed that the backpropagation algorithm is essential for learning good feature detectors in early layers of artificial neural networks, so that these detectors are useful for the task performed by the higher layers of that neural network. At the same time, the traditional form of backpropagation is biologically implausible. In the present paper we propose an unusual learning rule, which has a degree of biological plausibility, and which is motivated by Hebb's idea that change of the synapse strength should be local - i.e. should depend only on the activities of the pre and post synaptic neurons. We design a learning algorithm that utilizes global inhibition in the hidden layer, and is capable of learning early feature detectors in a completely unsupervised way. These learned lower layer feature detectors can be used to train higher layer weights in a usual supervised way so that the performance of the full network is comparable to the performance of standard feedforward networks trained end-to-end with a backpropagation algorithm.
Speeding Up Budgeted Stochastic Gradient Descent SVM Training with Precomputed Golden Section Search
Authors: Tobias Glasmachers, Sahar Qaadan
Date: 2018-06-26 00:00:00

Limiting the model size of a kernel support vector machine to a pre-defined
budget is a well-established technique that allows to scale SVM learning and
prediction to large-scale data. Its core addition to simple stochastic gradient
training is budget maintenance through merging of support vectors. This
requires solving an inner optimization problem with an iterative method many
times per gradient step. In this paper we replace the iterative procedure with
a fast lookup. We manage to reduce the merging time by up to 65% and the total
training time by 44% without any loss of accuracy.
Multi-Merge Budget Maintenance for Stochastic Gradient Descent SVM Training
Authors: Sahar Qaadan, Tobias Glasmachers
Date: 2018-06-26 00:00:00

Budgeted Stochastic Gradient Descent (BSGD) is a state-of-the-art technique
for training large-scale kernelized support vector machines. The budget
constraint is maintained incrementally by merging two points whenever the
pre-defined budget is exceeded. The process of finding suitable merge partners
is costly; it can account for up to 45% of the total training time. In this
paper we investigate computationally more efficient schemes that merge more
than two points at once. We obtain significant speed-ups without sacrificing
accuracy.
Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling
Authors: Shanshan Wu, Alexandros G. Dimakis, Sujay Sanghavi, Felix X. Yu, Daniel Holtmann-Rice, Dmitry Storcheus, Afshin Rostamizadeh, Sanjiv Kumar
Date: 2018-06-26 00:00:00
Tasks: compressed sensing, Decoder, Extreme Multi-Label Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Label Learning, Rolling Shutter Correction

Linear encoding of sparse vectors is widely popular, but is commonly data-independent -- missing any possible extra (but a priori unknown) structure beyond sparsity. In this paper we present a new method to learn linear encoders that adapt to data, while still performing well with the widely used $\ell_1$ decoder. The convex $\ell_1$ decoder prevents gradient propagation as needed in standard gradient-based training. Our method is based on the insight that unrolling the convex decoder into $T$ projected subgradient steps can address this issue. Our method can be seen as a data-driven way to learn a compressed sensing measurement matrix. We compare the empirical performance of 10 algorithms over 6 sparse datasets (3 synthetic and 3 real). Our experiments show that there is indeed additional structure beyond sparsity in the real datasets; our method is able to discover it and exploit it to create excellent reconstructions with fewer measurements (by a factor of 1.1-3x) compared to the previous state-of-the-art methods. We illustrate an application of our method in learning label embeddings for extreme multi-label classification, and empirically show that our method is able to match or outperform the precision scores of SLEEC, which is one of the state-of-the-art embedding-based approaches.
Semantically Enhanced Dynamic Bayesian Network for Detecting Sepsis Mortality Risk in ICU Patients with Infection
Authors: Tony Wang, Tom Velez, Emilia Apostolova, Tim Tschampel, Thuy L. Ngo, Joy Hardison
Date: 2018-06-26 00:00:00

Although timely sepsis diagnosis and prompt interventions in Intensive Care
Unit (ICU) patients are associated with reduced mortality, early clinical
recognition is frequently impeded by non-specific signs of infection and
failure to detect signs of sepsis-induced organ dysfunction in a constellation
of dynamically changing physiological data. The goal of this work is to
identify patient at risk of life-threatening sepsis utilizing a data-centered
and machine learning-driven approach. We derive a mortality risk predictive
dynamic Bayesian network (DBN) guided by a customized sepsis knowledgebase and
compare the predictive accuracy of the derived DBN with the Sepsis-related
Organ Failure Assessment (SOFA) score, the Quick SOFA (qSOFA) score, the
Simplified Acute Physiological Score (SAPS-II) and the Modified Early Warning
Score (MEWS) tools.
  A customized sepsis ontology was used to derive the DBN node structure and
semantically characterize temporal features derived from both structured
physiological data and unstructured clinical notes. We assessed the performance
in predicting mortality risk of the DBN predictive model and compared
performance to other models using Receiver Operating Characteristic (ROC)
curves, area under curve (AUROC), calibration curves, and risk distributions.
  The derived dataset consists of 24,506 ICU stays from 19,623 patients with
evidence of suspected infection, with 2,829 patients deceased at discharge. The
DBN AUROC was found to be 0.91, which outperformed the SOFA (0.843), qSOFA
(0.66), MEWS (0.73), and SAPS-II (0.77) scoring tools. Continuous Net
Reclassification Index and Integrated Discrimination Improvement analysis
supported the superiority DBN. Compared with conventional rule-based risk
scoring tools, the sepsis knowledgebase-driven DBN algorithm offers improved
performance for predicting mortality of infected patients in ICUs.
MMSE Approximation For Sparse Coding Algorithms Using Stochastic Resonance
Authors: Dror Simon, Jeremias Sulam, Yaniv Romano, Yue M. Lu, Michael Elad
Date: 2018-06-26 00:00:00

Sparse coding refers to the pursuit of the sparsest representation of a
signal in a typically overcomplete dictionary. From a Bayesian perspective,
sparse coding provides a Maximum a Posteriori (MAP) estimate of the unknown
vector under a sparse prior. In this work, we suggest enhancing the performance
of sparse coding algorithms by a deliberate and controlled contamination of the
input with random noise, a phenomenon known as stochastic resonance. The
proposed method adds controlled noise to the input and estimates a sparse
representation from the perturbed signal. A set of such solutions is then
obtained by projecting the original input signal onto the recovered set of
supports. We present two variants of the described method, which differ in
their final step. The first is a provably convergent approximation to the
Minimum Mean Square Error (MMSE) estimator, relying on the generative model and
applying a weighted average over the recovered solutions. The second is a
relaxed variant of the former that simply applies an empirical mean. We show
that both methods provide a computationally efficient approximation to the MMSE
estimator, which is typically intractable to compute. We demonstrate our
findings empirically and provide a theoretical analysis of our method under
several different cases.
Detection of Alzheimers Disease from MRI using Convolutional Neural Network with Tensorflow
Authors: GuruRaj Awate, Sunil Bangare, G Pradeepini, S. Patil
Date: 2018-06-26 00:00:00
Tasks: GPU

Nowadays, due to tremendous improvements in high performance computing, it has become easier to train Neural Networks. We intend to take advantage of this situation and apply this technology in solving real world problems. There was a need for automatic diagnosis certain diseases from medical images that could help a doctor and radiologist for further action towards treating the illness. We chose Alzheimer disease for this purpose. Alzheimer disease is the leading cause of dementia and memory loss. Alzheimer disease, it is caused by atrophy of the certain brain regions and by brain cell death. MRI scans reveal this information but atrophy regions are different for different people which makes the diagnosis a little trickier and often gets miss-diagnosed by doctors and radiologists. The Dataset used for this project is provided by OASIS, which contains over 400 subjects 100 of which having mild to severe dementia and is supplemented by MMSE and CDR standards of diagnosis in the same context. Enter CNN, Convolutional Neural Networks are a hybrid of Kernel Convolutions and Neural Networks. Kernel Convolutions is a technique that uses filters to recognize and segment images based on features. Neural Networks consist of neurons which are loosely based on human brains neuron which represents a single classifier and interconnected by weights, have different biases and are activated by some activation functions. By using Convolutional Neural Networks, the problem can be solved with minimal error rate. The technologies we intend to use are libraries like CUDA CuDNN for making use of GPU and its multiple cores-parallel computing to train models while giving us high performance.