pwc-archive/papers-with-abstracts huggingface dataset:
Reading Between the Pixels: Photographic Steganography for Camera Display Messaging
Authors: Eric Wengrowski, Kristin Dana, Marco Gruteser, Narayan Mandayam
Date: 2016-04-06 00:00:00
Tasks: Sensitivity

We exploit human color metamers to send light-modulated messages less visible
to the human eye, but recoverable by cameras. These messages are a key
component to camera-display messaging, such as handheld smartphones capturing
information from electronic signage. Each color pixel in the display image is
modified by a particular color gradient vector. The challenge is to find the
color gradient that maximizes camera response, while minimizing human response.
The mismatch in human spectral and camera sensitivity curves creates an
opportunity for hidden messaging. Our approach does not require knowledge of
these sensitivity curves, instead we employ a data-driven method. We learn an
ellipsoidal partitioning of the six-dimensional space of colors and color
gradients. This partitioning creates metamer sets defined by the base color at
the display pixel and the color gradient direction for message encoding. We
sample from the resulting metamer sets to find color steps for each base color
to embed a binary message into an arbitrary image with reduced visible
artifacts. Unlike previous methods that rely on visually obtrusive intensity
modulation, we embed with color so that the message is more hidden. Ordinary
displays and cameras are used without the need for expensive LEDs or high speed
devices. The primary contribution of this work is a framework to map the pixels
in an arbitrary image to a metamer pair for steganographic photo messaging.
Accelerating Science: A Computing Research Agenda
Authors: Vasant G. Honavar, Mark D. Hill, Katherine Yelick
Date: 2016-04-06 00:00:00
Tasks: Management, scientific discovery

The emergence of "big data" offers unprecedented opportunities for not only
accelerating scientific advances but also enabling new modes of discovery.
Scientific progress in many disciplines is increasingly enabled by our ability
to examine natural phenomena through the computational lens, i.e., using
algorithmic or information processing abstractions of the underlying processes;
and our ability to acquire, share, integrate and analyze disparate types of
data. However, there is a huge gap between our ability to acquire, store, and
process data and our ability to make effective use of the data to advance
discovery. Despite successful automation of routine aspects of data management
and analytics, most elements of the scientific process currently require
considerable human expertise and effort. Accelerating science to keep pace with
the rate of data acquisition and data processing calls for the development of
algorithmic or information processing abstractions, coupled with formal methods
and tools for modeling and simulation of natural processes as well as major
innovations in cognitive tools for scientists, i.e., computational tools that
leverage and extend the reach of human intellect, and partner with humans on a
broad range of tasks in scientific discovery (e.g., identifying, prioritizing
formulating questions, designing, prioritizing and executing experiments
designed to answer a chosen question, drawing inferences and evaluating the
results, and formulating new questions, in a closed-loop fashion). This calls
for concerted research agenda aimed at: Development, analysis, integration,
sharing, and simulation of algorithmic or information processing abstractions
of natural processes, coupled with formal methods and tools for their analyses
and simulation; Innovations in cognitive tools that augment and extend human
intellect and partner with humans in all aspects of science.
Learning in Auctions: Regret is Hard, Envy is Easy
Authors: Constantinos Daskalakis, Vasilis Syrgkanis
Date: 2015-11-04 00:00:00

A line of recent work provides welfare guarantees of simple combinatorial
auction formats, such as selling m items via simultaneous second price auctions
(SiSPAs) (Christodoulou et al. 2008, Bhawalkar and Roughgarden 2011, Feldman et
al. 2013). These guarantees hold even when the auctions are repeatedly executed
and players use no-regret learning algorithms. Unfortunately, off-the-shelf
no-regret algorithms for these auctions are computationally inefficient as the
number of actions is exponential. We show that this obstacle is insurmountable:
there are no polynomial-time no-regret algorithms for SiSPAs, unless
RP$\supseteq$ NP, even when the bidders are unit-demand. Our lower bound raises
the question of how good outcomes polynomially-bounded bidders may discover in
such auctions.
  To answer this question, we propose a novel concept of learning in auctions,
termed "no-envy learning." This notion is founded upon Walrasian equilibrium,
and we show that it is both efficiently implementable and results in
approximately optimal welfare, even when the bidders have fractionally
subadditive (XOS) valuations (assuming demand oracles) or coverage valuations
(without demand oracles). No-envy learning outcomes are a relaxation of
no-regret outcomes, which maintain their approximate welfare optimality while
endowing them with computational tractability. Our results extend to other
auction formats that have been studied in the literature via the smoothness
paradigm.
  Our results for XOS valuations are enabled by a novel
Follow-The-Perturbed-Leader algorithm for settings where the number of experts
is infinite, and the payoff function of the learner is non-linear. This
algorithm has applications outside of auction settings, such as in security
games. Our result for coverage valuations is based on a novel use of convex
rounding schemes and a reduction to online convex optimization.
On the Geometry of Message Passing Algorithms for Gaussian Reciprocal Processes
Authors: Francesca Paola Carli
Date: 2016-03-30 00:00:00

Reciprocal processes are acausal generalizations of Markov processes
introduced by Bernstein in 1932. In the literature, a significant amount of
attention has been focused on developing dynamical models for reciprocal
processes. Recently, probabilistic graphical models for reciprocal processes
have been provided. This opens the way to the application of efficient
inference algorithms in the machine learning literature to solve the smoothing
problem for reciprocal processes. Such algorithms are known to converge if the
underlying graph is a tree. This is not the case for a reciprocal process,
whose associated graphical model is a single loop network. The contribution of
this paper is twofold. First, we introduce belief propagation for Gaussian
reciprocal processes. Second, we establish a link between convergence analysis
of belief propagation for Gaussian reciprocal processes and stability theory
for differentially positive systems.
A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories
Authors: Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, James Allen
Date: 2016-04-06 00:00:00
Tasks: Cloze Test, Sentence, Story Generation

Representation and learning of commonsense knowledge is one of the
foundational problems in the quest to enable deep language understanding. This
issue is particularly challenging for understanding casual and correlational
relationships between events. While this topic has received a lot of interest
in the NLP community, research has been hindered by the lack of a proper
evaluation framework. This paper attempts to address this problem with a new
framework for evaluating story understanding and script learning: the 'Story
Cloze Test'. This test requires a system to choose the correct ending to a
four-sentence story. We created a new corpus of ~50k five-sentence commonsense
stories, ROCStories, to enable this evaluation. This corpus is unique in two
ways: (1) it captures a rich set of causal and temporal commonsense relations
between daily events, and (2) it is a high quality collection of everyday life
stories that can also be used for story generation. Experimental evaluation
shows that a host of baselines and state-of-the-art models based on shallow
language understanding struggle to achieve a high score on the Story Cloze
Test. We discuss these implications for script and story learning, and offer
suggestions for deeper language understanding.
An Ensemble Method to Produce High-Quality Word Embeddings (2016)
Authors: Robyn Speer, Joshua Chin
Date: 2016-04-06 00:00:00
Tasks: Vocal Bursts Intensity Prediction, Word Embeddings, Word Similarity

A currently successful approach to computational semantics is to represent words as embeddings in a machine-learned vector space. We present an ensemble method that combines embeddings produced by GloVe (Pennington et al., 2014) and word2vec (Mikolov et al., 2013) with structured knowledge from the semantic networks ConceptNet (Speer and Havasi, 2012) and PPDB (Ganitkevitch et al., 2013), merging their information into a common representation with a large, multilingual vocabulary. The embeddings it produces achieve state-of-the-art performance on many word-similarity evaluations. Its score of $\rho = .596$ on an evaluation of rare words (Luong et al., 2013) is 16% higher than the previous best known system.
Improving Back-Propagation by Adding an Adversarial Gradient
Authors: Arild Nøkland
Date: 2015-10-14 00:00:00
Tasks: BIG-bench Machine Learning, General Classification

The back-propagation algorithm is widely used for learning in artificial
neural networks. A challenge in machine learning is to create models that
generalize to new data samples not seen in the training data. Recently, a
common flaw in several machine learning algorithms was discovered: small
perturbations added to the input data lead to consistent misclassification of
data samples. Samples that easily mislead the model are called adversarial
examples. Training a "maxout" network on adversarial examples has shown to
decrease this vulnerability, but also increase classification performance. This
paper shows that adversarial training has a regularizing effect also in
networks with logistic, hyperbolic tangent and rectified linear units. A simple
extension to the back-propagation method is proposed, that adds an adversarial
gradient to the training. The extension requires an additional forward and
backward pass to calculate a modified input sample, or mini batch, used as
input for standard back-propagation learning. The first experimental results on
MNIST show that the "adversarial back-propagation" method increases the
resistance to adversarial examples and boosts the classification performance.
The extension reduces the classification error on the permutation invariant
MNIST from 1.60% to 0.95% in a logistic network, and from 1.40% to 0.78% in a
network with rectified linear units. Results on CIFAR-10 indicate that the
method has a regularizing effect similar to dropout in fully connected
networks. Based on these promising results, adversarial back-propagation is
proposed as a stand-alone regularizing method that should be further
investigated.
DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks
Authors: Jie Fu, Hongyin Luo, Jiashi Feng, Kian Hsiang Low, Tat-Seng Chua
Date: 2016-01-05 00:00:00

The performance of deep neural networks is well-known to be sensitive to the
setting of their hyperparameters. Recent advances in reverse-mode automatic
differentiation allow for optimizing hyperparameters with gradients. The
standard way of computing these gradients involves a forward and backward pass
of computations. However, the backward pass usually needs to consume
unaffordable memory to store all the intermediate variables to exactly reverse
the forward training procedure. In this work we propose a simple but effective
method, DrMAD, to distill the knowledge of the forward pass into a shortcut
path, through which we approximately reverse the training trajectory.
Experiments on several image benchmark datasets show that DrMAD is at least 45
times faster and consumes 100 times less memory compared to state-of-the-art
methods for optimizing hyperparameters with minimal compromise to its
effectiveness. To the best of our knowledge, DrMAD is the first research
attempt to make it practical to automatically tune thousands of hyperparameters
of deep neural networks. The code can be downloaded from
https://github.com/bigaidream-projects/drmad
Computing with hardware neurons: spiking or classical? Perspectives of applied Spiking Neural Networks from the hardware side
Authors: Sergei Dytckov, Masoud Daneshtalab
Date: 2016-02-05 00:00:00

While classical neural networks take a position of a leading method in the
machine learning community, spiking neuromorphic systems bring attention and
large projects in neuroscience. Spiking neural networks were shown to be able
to substitute networks of classical neurons in applied tasks. This work
explores recent hardware designs focusing on perspective applications (like
convolutional neural networks) for both neuron types from the energy efficiency
side to analyse whether there is a possibility for spiking neuromorphic
hardware to grow up for a wider use. Our comparison shows that spiking hardware
is at least on the same level of energy efficiency or even higher than
non-spiking on a level of basic operations. However, on a system level, spiking
systems are outmatched and consume much more energy due to inefficient data
representation with a long series of spikes. If spike-driven applications,
minimizing an amount of spikes, are developed, spiking neural systems may reach
the energy efficiency level of classical neural systems. However, in the near
future, both type of neuromorphic systems may benefit from emerging memory
technologies, minimizing the energy consumption of computation and memory for
both neuron types. That would make infrastructure and data transfer energy
dominant on the system level. We expect that spiking neurons have some
benefits, which would allow achieving better energy results. Still the problem
of an amount of spikes will still be the major bottleneck for spiking hardware
systems.
Modeling self-organization of vocabularies under phonological similarity effects
Authors: Javier Vera
Date: 2016-03-17 00:00:00

This work develops a computational model (by Automata Networks) of
phonological similarity effects involved in the formation of word-meaning
associations on artificial populations of speakers. Classical studies show that
in recalling experiments memory performance was impaired for phonologically
similar words versus dissimilar ones. Here, the individuals confound
phonologically similar words according to a predefined parameter. The main
hypothesis is that there is a critical range of the parameter, and with this,
of working-memory mechanisms, which implies drastic changes in the final
consensus of the entire population. Theoretical results present proofs of
convergence for a particular case of the model within a worst-case complexity
framework. Computer simulations describe the evolution of an energy function
that measures the amount of local agreement between individuals. The main
finding is the appearance of sudden changes in the energy function at critical
parameters.
A Self-Paced Regularization Framework for Multi-Label Learning
Authors: Changsheng Li, Fan Wei, Junchi Yan, Weishan Dong, Qingshan Liu, Xiao-Yu Zhang, Hongyuan Zha
Date: 2016-03-22 00:00:00
Tasks: Multi-Label Learning

In this paper, we propose a novel multi-label learning framework, called
Multi-Label Self-Paced Learning (MLSPL), in an attempt to incorporate the
self-paced learning strategy into multi-label learning regime. In light of the
benefits of adopting the easy-to-hard strategy proposed by self-paced learning,
the devised MLSPL aims to learn multiple labels jointly by gradually including
label learning tasks and instances into model training from the easy to the
hard. We first introduce a self-paced function as a regularizer in the
multi-label learning formulation, so as to simultaneously rank priorities of
the label learning tasks and the instances in each learning iteration.
Considering that different multi-label learning scenarios often need different
self-paced schemes during optimization, we thus propose a general way to find
the desired self-paced functions. Experimental results on three benchmark
datasets suggest the state-of-the-art performance of our approach.
Nonparametric Detection of Geometric Structures over Networks
Authors: Shaofeng Zou, Yingbin Liang, H. Vincent Poor
Date: 2016-04-05 00:00:00

Nonparametric detection of existence of an anomalous structure over a network
is investigated. Nodes corresponding to the anomalous structure (if one exists)
receive samples generated by a distribution q, which is different from a
distribution p generating samples for other nodes. If an anomalous structure
does not exist, all nodes receive samples generated by p. It is assumed that
the distributions p and q are arbitrary and unknown. The goal is to design
statistically consistent tests with probability of errors converging to zero as
the network size becomes asymptotically large. Kernel-based tests are proposed
based on maximum mean discrepancy that measures the distance between mean
embeddings of distributions into a reproducing kernel Hilbert space. Detection
of an anomalous interval over a line network is first studied. Sufficient
conditions on minimum and maximum sizes of candidate anomalous intervals are
characterized in order to guarantee the proposed test to be consistent. It is
also shown that certain necessary conditions must hold to guarantee any test to
be universally consistent. Comparison of sufficient and necessary conditions
yields that the proposed test is order-level optimal and nearly optimal
respectively in terms of minimum and maximum sizes of candidate anomalous
intervals. Generalization of the results to other networks is further
developed. Numerical results are provided to demonstrate the performance of the
proposed tests.
ASlib: A Benchmark Library for Algorithm Selection
Authors: Bernd Bischl, Pascal Kerschke, Lars Kotthoff, Marius Lindauer, Yuri Malitsky, Alexandre Frechette, Holger Hoos, Frank Hutter, Kevin Leyton-Brown, Kevin Tierney, Joaquin Vanschoren
Date: 2015-06-08 00:00:00

The task of algorithm selection involves choosing an algorithm from a set of
algorithms on a per-instance basis in order to exploit the varying performance
of algorithms over a set of instances. The algorithm selection problem is
attracting increasing attention from researchers and practitioners in AI. Years
of fruitful applications in a number of domains have resulted in a large amount
of data, but the community lacks a standard format or repository for this data.
This situation makes it difficult to share and compare different approaches
effectively, as is done in other, more established fields. It also
unnecessarily hinders new researchers who want to work in this area. To address
this problem, we introduce a standardized format for representing algorithm
selection scenarios and a repository that contains a growing number of data
sets from the literature. Our format has been designed to be able to express a
wide variety of different scenarios. Demonstrating the breadth and power of our
platform, we describe a set of example experiments that build and evaluate
algorithm selection models through a common interface. The results display the
potential of algorithm selection to achieve significant performance
improvements across a broad range of problems and algorithms.
Fast $(1+ε)$-approximation of the Löwner extremal matrices of high-dimensional symmetric matrices
Authors: Frank Nielsen, Richard Nock
Date: 2016-04-06 00:00:00
Tasks: Clustering

Matrix data sets are common nowadays like in biomedical imaging where the
Diffusion Tensor Magnetic Resonance Imaging (DT-MRI) modality produces data
sets of
  3D symmetric positive definite matrices anchored at voxel positions capturing
the anisotropic diffusion properties of water molecules in biological tissues.
The space of symmetric matrices can be partially ordered using the L\"owner
ordering, and computing extremal matrices dominating a given set of matrices is
a basic primitive used in matrix-valued signal processing. In this letter, we
design a fast and easy-to-implement iterative algorithm to approximate
arbitrarily finely these extremal matrices. Finally, we discuss on extensions
to matrix clustering.
Computational Sociolinguistics: A Survey
Authors: Dong Nguyen, A. Seza Doğruöz, Carolyn P. Rosé, Franciska de Jong
Date: 2015-08-30 00:00:00
Tasks: Survey

Language is a social phenomenon and variation is inherent to its social
nature. Recently, there has been a surge of interest within the computational
linguistics (CL) community in the social dimension of language. In this article
we present a survey of the emerging field of "Computational Sociolinguistics"
that reflects this increased interest. We aim to provide a comprehensive
overview of CL research on sociolinguistic themes, featuring topics such as the
relation between language and social identity, language use in social
interaction and multilingual communication. Moreover, we demonstrate the
potential for synergy between the research communities involved, by showing how
the large-scale data-driven methods that are widely used in CL can complement
existing sociolinguistic studies, and how sociolinguistics can inform and
challenge the methods and assumptions employed in CL studies. We hope to convey
the possible benefits of a closer collaboration between the two communities and
conclude with a discussion of open challenges.
Spiking Analog VLSI Neuron Assemblies as Constraint Satisfaction Problem Solvers
Authors: Jonathan Binas, Giacomo Indiveri, Michael Pfeiffer
Date: 2015-11-02 00:00:00

Solving constraint satisfaction problems (CSPs) is a notoriously expensive
computational task. Recently, it has been proposed that efficient stochastic
solvers can be obtained through appropriately configured spiking neural
networks performing Markov Chain Monte Carlo (MCMC) sampling. The possibility
to run such models on massively parallel, low-power neuromorphic hardware holds
great promise; however, previously proposed networks are based on
probabilistically spiking neurons, and thus rely on random number generators or
external noise sources to achieve the necessary stochasticity, leading to
significant overhead in the implementation. Here we show how stochasticity can
be achieved by implementing deterministic models of integrate and fire neurons
using subthreshold analog circuits that are affected by thermal noise. We
present an efficient implementation of spike-based CSP solvers using a
reconfigurable neural network VLSI device, and the device's intrinsic noise as
a source of randomness. To illustrate the overall concept, we implement a
generic Sudoku solver based on our approach and demonstrate its operation. We
establish a link between the neuron parameters and the system dynamics,
allowing for a simple temperature control mechanism.