pwc-archive/papers-with-abstracts huggingface dataset:
Color-opponent mechanisms for local hue encoding in a hierarchical framework
Authors: Paria Mehrani, Andrei Mouraviev, Oscar J. Avella Gonzalez, John K. Tsotsos
Date: 2017-06-30 00:00:00

A biologically plausible computational model for color representation is
introduced. We present a mechanistic hierarchical model of neurons that not
only successfully encodes local hue, but also explicitly reveals how the
contributions of each visual cortical layer participating in the process can
lead to a hue representation. Our proposed model benefits from studies on the
visual cortex and builds a network of single-opponent and hue-selective
neurons. Local hue encoding is achieved through gradually increasing
nonlinearity in terms of cone inputs to single-opponent cells. We demonstrate
that our model's single-opponent neurons have wide tuning curves, while the
hue-selective neurons in our model V4 layer exhibit narrower tunings,
resembling those in V4 of the primate visual system. Our simulation experiments
suggest that neurons in V4 or later layers have the capacity of encoding unique
hues. Moreover, with a few examples, we present the possibility of spanning the
infinite space of physical hues by combining the hue-selective neurons in our
model.
Actively Avoiding Nonsense in Generative Models
Authors: Steve Hanneke, Adam Kalai, Gautam Kamath, Christos Tzamos
Date: 2018-02-20 00:00:00

A generative model may generate utter nonsense when it is fit to maximize the
likelihood of observed data. This happens due to "model error," i.e., when the
true data generating distribution does not fit within the class of generative
models being learned. To address this, we propose a model of active
distribution learning using a binary invalidity oracle that identifies some
examples as clearly invalid, together with random positive examples sampled
from the true distribution. The goal is to maximize the likelihood of the
positive examples subject to the constraint of (almost) never generating
examples labeled invalid by the oracle. Guarantees are agnostic compared to a
class of probability distributions. We show that, while proper learning often
requires exponentially many queries to the invalidity oracle, improper
distribution learning can be done using polynomially many queries.
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Authors: Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, Hyrum Anderson, Heather Roff, Gregory C. Allen, Jacob Steinhardt, Carrick Flynn, Seán Ó hÉigeartaigh, SJ Beard, Haydn Belfield, Sebastian Farquhar, Clare Lyle, Rebecca Crootof, Owain Evans, Michael Page, Joanna Bryson, Roman Yampolskiy, Dario Amodei
Date: 2018-02-20 00:00:00

This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats. After analyzing the ways in which AI may influence the threat landscape in the digital, physical, and political domains, we make four high-level recommendations for AI researchers and other stakeholders. We also suggest several promising areas for further research that could expand the portfolio of defenses, or make attacks less effective or harder to execute. Finally, we discuss, but do not conclusively resolve, the long-term equilibrium of attackers and defenders.
Spurious Valleys in Two-layer Neural Network Optimization Landscapes
Authors: Luca Venturi, Afonso S. Bandeira, Joan Bruna
Date: 2018-02-18 00:00:00
Tasks: Vocal Bursts Valence Prediction

Neural networks provide a rich class of high-dimensional, non-convex optimization problems. Despite their non-convexity, gradient-descent methods often successfully optimize these models. This has motivated a recent spur in research attempting to characterize properties of their loss surface that may explain such success. In this paper, we address this phenomenon by studying a key topological property of the loss: the presence or absence of spurious valleys, defined as connected components of sub-level sets that do not include a global minimum. Focusing on a class of two-layer neural networks defined by smooth (but generally non-linear) activation functions, we identify a notion of intrinsic dimension and show that it provides necessary and sufficient conditions for the absence of spurious valleys. More concretely, finite intrinsic dimension guarantees that for sufficiently overparametrised models no spurious valleys exist, independently of the data distribution. Conversely, infinite intrinsic dimension implies that spurious valleys do exist for certain data distributions, independently of model overparametrisation. Besides these positive and negative results, we show that, although spurious valleys may exist in general, they are confined to low risk levels and avoided with high probability on overparametrised models.
Stability of meanings versus rate of replacement of words: an experimental test
Authors: Michele Pasquini, Maurizio Serva
Date: 2018-02-20 00:00:00

The words of a language are randomly replaced in time by new ones, but it has
long been known that words corresponding to some items (meanings) are less
frequently replaced than others. Usually, the rate of replacement for a given
item is not directly observable, but it is inferred by the estimated stability
which, on the contrary, is observable. This idea goes back a long way in the
lexicostatistical literature, nevertheless nothing ensures that it gives the
correct answer. The family of Romance languages allows for a direct test of the
estimated stabilities against the replacement rates since the proto-language
(Latin) is known and the replacement rates can be explicitly computed. The
output of the test is threefold:first, we prove that the standard approach
which tries to infer the replacement rates trough the estimated stabilities is
sound; second, we are able to rewrite the fundamental formula of
Glottochronology for a non universal replacement rate (a rate which depends on
the item); third, we give indisputable evidence that the stability ranking is
far from being the same for different families of languages. This last result
is also supported by comparison with the Malagasy family of dialects. As a side
result we also provide some evidence that Vulgar Latin and not Late Classical
Latin is at the root of modern Romance languages.
Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling
Authors: Arka Daw, Anuj Karpatne, William Watkins, Jordan Read, Vipin Kumar
Date: 2017-10-31 00:00:00
Tasks: scientific discovery

This paper introduces a framework for combining scientific knowledge of physics-based models with neural networks to advance scientific discovery. This framework, termed physics-guided neural networks (PGNN), leverages the output of physics-based model simulations along with observational features in a hybrid modeling setup to generate predictions using a neural network architecture. Further, this framework uses physics-based loss functions in the learning objective of neural networks to ensure that the model predictions not only show lower errors on the training set but are also scientifically consistent with the known physics on the unlabeled set. We illustrate the effectiveness of PGNN for the problem of lake temperature modeling, where physical relationships between the temperature, density, and depth of water are used to design a physics-based loss function. By using scientific knowledge to guide the construction and learning of neural networks, we are able to show that the proposed framework ensures better generalizability as well as scientific consistency of results. All the code and datasets used in this study have been made available on this link \url{https://github.com/arkadaw9/PGNN}.
What is Wrong with Topic Modeling? (and How to Fix it Using Search-based Software Engineering)
Authors: Amritanshu Agrawal, Wei Fu, Tim Menzies
Date: 2016-08-29 00:00:00
Tasks: General Classification

Context: Topic modeling finds human-readable structures in unstructured
textual data. A widely used topic modeler is Latent Dirichlet allocation. When
run on different datasets, LDA suffers from "order effects" i.e. different
topics are generated if the order of training data is shuffled. Such order
effects introduce a systematic error for any study. This error can relate to
misleading results;specifically, inaccurate topic descriptions and a reduction
in the efficacy of text mining classification results. Objective: To provide a
method in which distributions generated by LDA are more stable and can be used
for further analysis. Method: We use LDADE, a search-based software engineering
tool that tunes LDA's parameters using DE (Differential Evolution). LDADE is
evaluated on data from a programmer information exchange site (Stackoverflow),
title and abstract text of thousands ofSoftware Engineering (SE) papers, and
software defect reports from NASA. Results were collected across different
implementations of LDA (Python+Scikit-Learn, Scala+Spark); across different
platforms (Linux, Macintosh) and for different kinds of LDAs (VEM,or using
Gibbs sampling). Results were scored via topic stability and text mining
classification accuracy. Results: In all treatments: (i) standard LDA exhibits
very large topic instability; (ii) LDADE's tunings dramatically reduce cluster
instability; (iii) LDADE also leads to improved performances for supervised as
well as unsupervised learning. Conclusion: Due to topic instability, using
standard LDA with its "off-the-shelf" settings should now be depreciated. Also,
in future, we should require SE papers that use LDA to test and (if needed)
mitigate LDA topic instability. Finally, LDADE is a candidate technology for
effectively and efficiently reducing that instability.
Real-Time Dense Stereo Matching With ELAS on FPGA Accelerated Embedded Devices
Authors: Oscar Rahnama, Duncan Frost, Ondrej Miksik, Philip H. S. Torr
Date: 2018-02-20 00:00:00
Tasks: CPU, Stereo Matching, Stereo Matching Hand

For many applications in low-power real-time robotics, stereo cameras are the
sensors of choice for depth perception as they are typically cheaper and more
versatile than their active counterparts. Their biggest drawback, however, is
that they do not directly sense depth maps; instead, these must be estimated
through data-intensive processes. Therefore, appropriate algorithm selection
plays an important role in achieving the desired performance characteristics.
  Motivated by applications in space and mobile robotics, we implement and
evaluate a FPGA-accelerated adaptation of the ELAS algorithm. Despite offering
one of the best trade-offs between efficiency and accuracy, ELAS has only been
shown to run at 1.5-3 fps on a high-end CPU. Our system preserves all
intriguing properties of the original algorithm, such as the slanted plane
priors, but can achieve a frame rate of 47fps whilst consuming under 4W of
power. Unlike previous FPGA based designs, we take advantage of both components
on the CPU/FPGA System-on-Chip to showcase the strategy necessary to accelerate
more complex and computationally diverse algorithms for such low power,
real-time systems.
Stealing Hyperparameters in Machine Learning
Authors: Binghui Wang, Neil Zhenqiang Gong
Date: 2018-02-14 00:00:00
Tasks: BIG-bench Machine Learning, regression

Hyperparameters are critical in machine learning, as different hyperparameters often result in models with significantly different performance. Hyperparameters may be deemed confidential because of their commercial value and the confidentiality of the proprietary algorithms that the learner uses to learn them. In this work, we propose attacks on stealing the hyperparameters that are learned by a learner. We call our attacks hyperparameter stealing attacks. Our attacks are applicable to a variety of popular machine learning algorithms such as ridge regression, logistic regression, support vector machine, and neural network. We evaluate the effectiveness of our attacks both theoretically and empirically. For instance, we evaluate our attacks on Amazon Machine Learning. Our results demonstrate that our attacks can accurately steal hyperparameters. We also study countermeasures. Our results highlight the need for new defenses against our hyperparameter stealing attacks for certain machine learning algorithms.
AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learning
Authors: Ahmed M. Alaa, Mihaela van der Schaar
Date: 2018-02-20 00:00:00
Proceeding: ICML 2018
Tasks: Bayesian Optimization, Diagnostic, Meta-Learning, Prognosis

Clinical prognostic models derived from largescale healthcare data can inform
critical diagnostic and therapeutic decisions. To enable off-theshelf usage of
machine learning (ML) in prognostic research, we developed AUTOPROGNOSIS: a
system for automating the design of predictive modeling pipelines tailored for
clinical prognosis. AUTOPROGNOSIS optimizes ensembles of pipeline
configurations efficiently using a novel batched Bayesian optimization (BO)
algorithm that learns a low-dimensional decomposition of the pipelines
high-dimensional hyperparameter space in concurrence with the BO procedure.
This is achieved by modeling the pipelines performances as a black-box function
with a Gaussian process prior, and modeling the similarities between the
pipelines baseline algorithms via a sparse additive kernel with a Dirichlet
prior. Meta-learning is used to warmstart BO with external data from similar
patient cohorts by calibrating the priors using an algorithm that mimics the
empirical Bayes method. The system automatically explains its predictions by
presenting the clinicians with logical association rules that link patients
features to predicted risk strata. We demonstrate the utility of AUTOPROGNOSIS
using 10 major patient cohorts representing various aspects of cardiovascular
patient care.
A Unified Framework for Low-Rank plus Sparse Matrix Recovery
Authors: Xiao Zhang, Lingxiao Wang, Quanquan Gu
Date: 2017-02-21 00:00:00

We propose a unified framework to solve general low-rank plus sparse matrix
recovery problems based on matrix factorization, which covers a broad family of
objective functions satisfying the restricted strong convexity and smoothness
conditions. Based on projected gradient descent and the double thresholding
operator, our proposed generic algorithm is guaranteed to converge to the
unknown low-rank and sparse matrices at a locally linear rate, while matching
the best-known robustness guarantee (i.e., tolerance for sparsity). At the core
of our theory is a novel structural Lipschitz gradient condition for low-rank
plus sparse matrices, which is essential for proving the linear convergence
rate of our algorithm, and we believe is of independent interest to prove fast
rates for general superposition-structured models. We illustrate the
application of our framework through two concrete examples: robust matrix
sensing and robust PCA. Experiments on both synthetic and real datasets
corroborate our theory.
Few-Shot Learning with Graph Neural Networks
Authors: Victor Garcia, Joan Bruna
Date: 2017-11-10 00:00:00
Tasks: Active Learning, Cross-Domain Few-Shot, Few-Shot Learning, Graph Neural Network

We propose to study the problem of few-shot learning with the prism of
inference on a partially observed graphical model, constructed from a
collection of input images whose label can be either observed or not. By
assimilating generic message-passing inference algorithms with their
neural-network counterparts, we define a graph neural network architecture that
generalizes several of the recently proposed few-shot learning models. Besides
providing improved numerical performance, our framework is easily extended to
variants of few-shot learning, such as semi-supervised or active learning,
demonstrating the ability of graph-based models to operate well on 'relational'
tasks.
Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer
Authors: David Madras, Toniann Pitassi, Richard Zemel
Date: 2017-11-17 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Decision Making, Fairness

In many machine learning applications, there are multiple decision-makers
involved, both automated and human. The interaction between these agents often
goes unaddressed in algorithmic development. In this work, we explore a simple
version of this interaction with a two-stage framework containing an automated
model and an external decision-maker. The model can choose to say "Pass", and
pass the decision downstream, as explored in rejection learning. We extend this
concept by proposing "learning to defer", which generalizes rejection learning
by considering the effect of other agents in the decision-making process. We
propose a learning algorithm which accounts for potential biases held by
external decision-makers in a system. Experiments demonstrate that learning to
defer can make systems not only more accurate but also less biased. Even when
working with inconsistent or biased users, we show that deferring models still
greatly improve the accuracy and/or fairness of the entire system.
Approximating Partition Functions in Constant Time
Authors: Vishesh Jain, Frederic Koehler, Elchanan Mossel
Date: 2017-11-05 00:00:00

We study approximations of the partition function of dense graphical models.
Partition functions of graphical models play a fundamental role is statistical
physics, in statistics and in machine learning. Two of the main methods for
approximating the partition function are Markov Chain Monte Carlo and
Variational Methods. An impressive body of work in mathematics, physics and
theoretical computer science provides conditions under which Markov Chain Monte
Carlo methods converge in polynomial time. These methods often lead to
polynomial time approximation algorithms for the partition function in cases
where the underlying model exhibits correlation decay. There are very few
theoretical guarantees for the performance of variational methods. One
exception is recent results by Risteski (2016) who considered dense graphical
models and showed that using variational methods, it is possible to find an
$O(\epsilon n)$ additive approximation to the log partition function in time
$n^{O(1/\epsilon^2)}$ even in a regime where correlation decay does not hold.
  We show that under essentially the same conditions, an $O(\epsilon n)$
additive approximation of the log partition function can be found in constant
time, independent of $n$. In particular, our results cover dense Ising and
Potts models as well as dense graphical models with $k$-wise interaction. They
also apply for low threshold rank models.
Adaptive Sampling for Coarse Ranking
Authors: Sumeet Katariya, Lalit Jain, Nandana Sengupta, James Evans, Robert Nowak
Date: 2018-02-20 00:00:00

We consider the problem of active coarse ranking, where the goal is to sort
items according to their means into clusters of pre-specified sizes, by
adaptively sampling from their reward distributions. This setting is useful in
many social science applications involving human raters and the approximate
rank of every item is desired. Approximate or coarse ranking can significantly
reduce the number of ratings required in comparison to the number needed to
find an exact ranking. We propose a computationally efficient PAC algorithm
LUCBRank for coarse ranking, and derive an upper bound on its sample
complexity. We also derive a nearly matching distribution-dependent lower
bound. Experiments on synthetic as well as real-world data show that LUCBRank
performs better than state-of-the-art baseline methods, even when these methods
have the advantage of knowing the underlying parametric model.
Iterative Refinement of the Approximate Posterior for Directed Belief Networks
Authors: R. Devon Hjelm, Kyunghyun Cho, Junyoung Chung, Russ Salakhutdinov, Vince Calhoun, Nebojsa Jojic
Date: 2015-11-19 00:00:00
Proceeding: NeurIPS 2016 12

Variational methods that rely on a recognition network to approximate the
posterior of directed graphical models offer better inference and learning than
previous methods. Recent advances that exploit the capacity and flexibility in
this approach have expanded what kinds of models can be trained. However, as a
proposal for the posterior, the capacity of the recognition network is limited,
which can constrain the representational power of the generative model and
increase the variance of Monte Carlo estimates. To address these issues, we
introduce an iterative refinement procedure for improving the approximate
posterior of the recognition network and show that training with the refined
posterior is competitive with state-of-the-art methods. The advantages of
refinement are further evident in an increased effective sample size, which
implies a lower variance of gradient estimates.
Bayesian Models for Unit Discovery on a Very Low Resource Language
Authors: Lucas Ondel, Pierre Godard, Laurent Besacier, Elin Larsen, Mark Hasegawa-Johnson, Odette Scharenborg, Emmanuel Dupoux, Lukas Burget, François Yvon, Sanjeev Khudanpur
Date: 2018-02-16 00:00:00
Tasks: Acoustic Unit Discovery, Segmentation

Developing speech technologies for low-resource languages has become a very
active research field over the last decade. Among others, Bayesian models have
shown some promising results on artificial examples but still lack of in situ
experiments. Our work applies state-of-the-art Bayesian models to unsupervised
Acoustic Unit Discovery (AUD) in a real low-resource language scenario. We also
show that Bayesian models can naturally integrate information from other
resourceful languages by means of informative prior leading to more consistent
discovered units. Finally, discovered acoustic units are used, either as the
1-best sequence or as a lattice, to perform word segmentation. Word
segmentation results show that this Bayesian approach clearly outperforms a
Segmental-DTW baseline on the same corpus.
One-Shot Coresets: The Case of k-Clustering
Authors: Olivier Bachem, Mario Lucic, Silvio Lattanzi
Date: 2017-11-27 00:00:00
Tasks: Clustering, Data Summarization, Small Data Image Classification

Scaling clustering algorithms to massive data sets is a challenging task.
Recently, several successful approaches based on data summarization methods,
such as coresets and sketches, were proposed. While these techniques provide
provably good and small summaries, they are inherently problem dependent - the
practitioner has to commit to a fixed clustering objective before even
exploring the data. However, can one construct small data summaries for a wide
range of clustering problems simultaneously? In this work, we affirmatively
answer this question by proposing an efficient algorithm that constructs such
one-shot summaries for k-clustering problems while retaining strong theoretical
guarantees.
Proximal Backpropagation
Authors: Thomas Frerix, Thomas Möllenhoff, Michael Moeller, Daniel Cremers
Date: 2017-06-14 00:00:00
Proceeding: ICLR 2018 1

We propose proximal backpropagation (ProxProp) as a novel algorithm that
takes implicit instead of explicit gradient steps to update the network
parameters during neural network training. Our algorithm is motivated by the
step size limitation of explicit gradient descent, which poses an impediment
for optimization. ProxProp is developed from a general point of view on the
backpropagation algorithm, currently the most common technique to train neural
networks via stochastic gradient descent and variants thereof. Specifically, we
show that backpropagation of a prediction error is equivalent to sequential
gradient descent steps on a quadratic penalty energy, which comprises the
network activations as variables of the optimization. We further analyze
theoretical properties of ProxProp and in particular prove that the algorithm
yields a descent direction in parameter space and can therefore be combined
with a wide variety of convergent algorithms. Finally, we devise an efficient
numerical implementation that integrates well with popular deep learning
frameworks. We conclude by demonstrating promising numerical results and show
that ProxProp can be effectively combined with common first order optimizers
such as Adam.