pwc-archive/papers-with-abstracts huggingface dataset:
The Limitations of Optimization from Samples
Authors: Eric Balkanski, Aviad Rubinstein, Yaron Singer
Date: 2015-12-19 00:00:00

In this paper we consider the following question: can we optimize objective
functions from the training data we use to learn them? We formalize this
question through a novel framework we call optimization from samples (OPS). In
OPS, we are given sampled values of a function drawn from some distribution and
the objective is to optimize the function under some constraint.
  While there are interesting classes of functions that can be optimized from
samples, our main result is an impossibility. We show that there are classes of
functions which are statistically learnable and optimizable, but for which no
reasonable approximation for optimization from samples is achievable. In
particular, our main result shows that there is no constant factor
approximation for maximizing coverage functions under a cardinality constraint
using polynomially-many samples drawn from any distribution.
  We also show tight approximation guarantees for maximization under a
cardinality constraint of several interesting classes of functions including
unit-demand, additive, and general monotone submodular functions, as well as a
constant factor approximation for monotone submodular functions with bounded
curvature.
Inferring Logical Forms From Denotations
Authors: Panupong Pasupat, Percy Liang
Date: 2016-06-22 00:00:00
Proceeding: ACL 2016 8

A core problem in learning semantic parsers from denotations is picking out
consistent logical forms--those that yield the correct denotation--from a
combinatorially large space. To control the search space, previous work relied
on restricted set of rules, which limits expressivity. In this paper, we
consider a much more expressive class of logical forms, and show how to use
dynamic programming to efficiently represent the complete set of consistent
logical forms. Expressivity also introduces many more spurious logical forms
which are consistent with the correct denotation but do not represent the
meaning of the utterance. To address this, we generate fictitious worlds and
use crowdsourced denotations on these worlds to filter out spurious logical
forms. On the WikiTableQuestions dataset, we increase the coverage of
answerable questions from 53.5% to 76%, and the additional crowdsourced
supervision lets us rule out 92.1% of spurious logical forms.
PixelVAE: A Latent Variable Model for Natural Images
Authors: Ishaan Gulrajani, Kundan Kumar, Faruk Ahmed, Adrien Ali Taiga, Francesco Visin, David Vazquez, Aaron Courville
Date: 2016-11-15 00:00:00
Tasks: Decoder, model

Natural image modeling is a landmark challenge of unsupervised learning.
Variational Autoencoders (VAEs) learn a useful latent representation and model
global structure well but have difficulty capturing small details. PixelCNN
models details very well, but lacks a latent code and is difficult to scale for
capturing large structures. We present PixelVAE, a VAE model with an
autoregressive decoder based on PixelCNN. Our model requires very few expensive
autoregressive layers compared to PixelCNN and learns latent codes that are
more compressed than a standard VAE while still capturing most non-trivial
structure. Finally, we extend our model to a hierarchy of latent variables at
different scales. Our model achieves state-of-the-art performance on binarized
MNIST, competitive performance on 64x64 ImageNet, and high-quality samples on
the LSUN bedrooms dataset.
Anchor-Free Correlated Topic Modeling: Identifiability and Algorithm
Authors: Kejun Huang, Xiao Fu, Nicholas D. Sidiropoulos
Date: 2016-11-15 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Clustering

In topic modeling, many algorithms that guarantee identifiability of the
topics have been developed under the premise that there exist anchor words --
i.e., words that only appear (with positive probability) in one topic.
Follow-up work has resorted to three or higher-order statistics of the data
corpus to relax the anchor word assumption. Reliable estimates of higher-order
statistics are hard to obtain, however, and the identification of topics under
those models hinges on uncorrelatedness of the topics, which can be
unrealistic. This paper revisits topic modeling based on second-order moments,
and proposes an anchor-free topic mining framework. The proposed approach
guarantees the identification of the topics under a much milder condition
compared to the anchor-word assumption, thereby exhibiting much better
robustness in practice. The associated algorithm only involves one
eigen-decomposition and a few small linear programs. This makes it easy to
implement and scale up to very large problem instances. Experiments using the
TDT2 and Reuters-21578 corpus demonstrate that the proposed anchor-free
approach exhibits very favorable performance (measured using coherence,
similarity count, and clustering accuracy metrics) compared to the prior art.
Gradients of Counterfactuals
Authors: Mukund Sundararajan, Ankur Taly, Qiqi Yan
Date: 2016-11-08 00:00:00
Tasks: counterfactual, Feature Importance, Language Modeling, Language Modelling, Object Recognition

Gradients have been used to quantify feature importance in machine learning
models. Unfortunately, in nonlinear deep networks, not only individual neurons
but also the whole network can saturate, and as a result an important input
feature can have a tiny gradient. We study various networks, and observe that
this phenomena is indeed widespread, across many inputs.
  We propose to examine interior gradients, which are gradients of
counterfactual inputs constructed by scaling down the original input. We apply
our method to the GoogleNet architecture for object recognition in images, as
well as a ligand-based virtual screening network with categorical features and
an LSTM based language model for the Penn Treebank dataset. We visualize how
interior gradients better capture feature importance. Furthermore, interior
gradients are applicable to a wide variety of deep networks, and have the
attribution property that the feature importance scores sum to the the
prediction score.
  Best of all, interior gradients can be computed just as easily as gradients.
In contrast, previous methods are complex to implement, which hinders practical
adoption.
Light Field Stitching for Extended Synthetic Aperture
Authors: M. Umair Mukati, Bahadir K. Gunturk
Date: 2016-11-15 00:00:00
Tasks: Depth Estimation

Through capturing spatial and angular radiance distribution, light field
cameras introduce new capabilities that are not possible with conventional
cameras. So far in the light field imaging literature, the focus has been on
the theory and applications of single light field capture. By combining
multiple light fields, it is possible to obtain new capabilities and
enhancements, and even exceed physical limitations, such as spatial resolution
and aperture size of the imaging device. In this paper, we present an algorithm
to register and stitch multiple light fields. We utilize the regularity of the
spatial and angular sampling in light field data, and extend some techniques
developed for stereo vision systems to light field data. Such an extension is
not straightforward for a micro-lens array (MLA) based light field camera due
to extremely small baseline and low spatial resolution. By merging multiple
light fields captured by an MLA based camera, we obtain larger synthetic
aperture, which results in improvements in light field capabilities, such as
increased depth estimation range/accuracy and wider perspective shift range.
Improved Multi-Class Cost-Sensitive Boosting via Estimation of the Minimum-Risk Class
Authors: Ron Appel, Xavier Burgos-Artizzu, Pietro Perona
Date: 2016-07-12 00:00:00

We present a simple unified framework for multi-class cost-sensitive
boosting. The minimum-risk class is estimated directly, rather than via an
approximation of the posterior distribution. Our method jointly optimizes
binary weak learners and their corresponding output vectors, requiring classes
to share features at each iteration. By training in a cost-sensitive manner,
weak learners are invested in separating classes whose discrimination is
important, at the expense of less relevant classification boundaries.
Additional contributions are a family of loss functions along with proof that
our algorithm is Boostable in the theoretical sense, as well as an efficient
procedure for growing decision trees for use as weak learners. We evaluate our
method on a variety of datasets: a collection of synthetic planar data, common
UCI datasets, MNIST digits, SUN scenes, and CUB-200 birds. Results show
state-of-the-art performance across all datasets against several strong
baselines, including non-boosting multi-class approaches.
An integrated Graphical User Interface for Debugging Answer Set Programs
Authors: Philip Gasteiger, Carmine Dodaro, Benjamin Musitsch, Kristian Reale, Francesco Ricca, Konstantin Schekotihin
Date: 2016-11-15 00:00:00

Answer Set Programming (ASP) is an expressive knowledge representation and
reasoning framework. Due to its rather simple syntax paired with
high-performance solvers, ASP is interesting for industrial applications.
However, to err is human and thus debugging is an important activity during the
development process. Therefore, tools for debugging non-ground answer set
programs are needed. In this paper, we present a new graphical debugging
interface for non-ground answer set programs. The tool is based on the
recently-introduced DWASP approach for debugging and it simplifies the
interaction with the debugger. Furthermore, the debugging interface is
integrated in ASPIDE, a rich IDE for answer set programs. With our extension
ASPIDE turns into a full-fledged IDE by offering debugging support.
Iterative Orthogonal Feature Projection for Diagnosing Bias in Black-Box Models
Authors: Julius Adebayo, Lalana Kagal
Date: 2016-11-15 00:00:00
Tasks: Fairness

Predictive models are increasingly deployed for the purpose of determining
access to services such as credit, insurance, and employment. Despite potential
gains in productivity and efficiency, several potential problems have yet to be
addressed, particularly the potential for unintentional discrimination. We
present an iterative procedure, based on orthogonal projection of input
attributes, for enabling interpretability of black-box predictive models.
Through our iterative procedure, one can quantify the relative dependence of a
black-box model on its input attributes.The relative significance of the inputs
to a predictive model can then be used to assess the fairness (or
discriminatory extent) of such a model.
Learning recurrent representations for hierarchical behavior modeling
Authors: Eyrun Eyjolfsdottir, Kristin Branson, Yisong Yue, Pietro Perona
Date: 2016-11-01 00:00:00
Tasks: Action Detection, motion prediction

We propose a framework for detecting action patterns from motion sequences
and modeling the sensory-motor relationship of animals, using a generative
recurrent neural network. The network has a discriminative part (classifying
actions) and a generative part (predicting motion), whose recurrent cells are
laterally connected, allowing higher levels of the network to represent high
level phenomena. We test our framework on two types of data, fruit fly behavior
and online handwriting. Our results show that 1) taking advantage of unlabeled
sequences, by predicting future motion, significantly improves action detection
performance when training labels are scarce, 2) the network learns to represent
high level phenomena such as writer identity and fly gender, without
supervision, and 3) simulated motion trajectories, generated by treating motion
prediction as input to the network, look realistic and may be used to
qualitatively evaluate whether the model has learnt generative control rules.
CIFAR-10: KNN-based Ensemble of Classifiers
Authors: Yehya Abouelnaga, Ola S. Ali, Hager Rady, Mohamed Moustafa
Date: 2016-11-15 00:00:00

In this paper, we study the performance of different classifiers on the
CIFAR-10 dataset, and build an ensemble of classifiers to reach a better
performance. We show that, on CIFAR-10, K-Nearest Neighbors (KNN) and
Convolutional Neural Network (CNN), on some classes, are mutually exclusive,
thus yield in higher accuracy when combined. We reduce KNN overfitting using
Principal Component Analysis (PCA), and ensemble it with a CNN to increase its
accuracy. Our approach improves our best CNN model from 93.33% to 94.03%.
Diversity encouraged learning of unsupervised LSTM ensemble for neural activity video prediction
Authors: Yilin Song, Jonathan Viventi, Yao Wang
Date: 2016-11-15 00:00:00
Tasks: Activity Prediction, Clustering, Diversity, Prediction, Video Prediction

Being able to predict the neural signal in the near future from the current
and previous observations has the potential to enable real-time responsive
brain stimulation to suppress seizures. We have investigated how to use an
auto-encoder model consisting of LSTM cells for such prediction. Recog- nizing
that there exist multiple activity pattern clusters, we have further explored
to train an ensemble of LSTM mod- els so that each model can specialize in
modeling certain neural activities, without explicitly clustering the training
data. We train the ensemble using an ensemble-awareness loss, which jointly
solves the model assignment problem and the error minimization problem. During
training, for each training sequence, only the model that has the lowest recon-
struction and prediction error is updated. Intrinsically such a loss function
enables each LTSM model to be adapted to a subset of the training sequences
that share similar dynamic behavior. We demonstrate this can be trained in an
end- to-end manner and achieve significant accuracy in neural activity
prediction.
Interpreting the Syntactic and Social Elements of the Tweet Representations via Elementary Property Prediction Tasks
Authors: J Ganesh, Manish Gupta, Vasudeva Varma
Date: 2016-11-15 00:00:00
Tasks: Property Prediction, Representation Learning, Semantic Textual Similarity, Sentiment Analysis

Research in social media analysis is experiencing a recent surge with a large
number of works applying representation learning models to solve high-level
syntactico-semantic tasks such as sentiment analysis, semantic textual
similarity computation, hashtag prediction and so on. Although the performance
of the representation learning models are better than the traditional baselines
for the tasks, little is known about the core properties of a tweet encoded
within the representations. Understanding these core properties would empower
us in making generalizable conclusions about the quality of representations.
Our work presented here constitutes the first step in opening the black-box of
vector embedding for social media posts, with emphasis on tweets in particular.
  In order to understand the core properties encoded in a tweet representation,
we evaluate the representations to estimate the extent to which it can model
each of those properties such as tweet length, presence of words, hashtags,
mentions, capitalization, and so on. This is done with the help of multiple
classifiers which take the representation as input. Essentially, each
classifier evaluates one of the syntactic or social properties which are
arguably salient for a tweet. This is also the first holistic study on
extensively analysing the ability to encode these properties for a wide variety
of tweet representation models including the traditional unsupervised methods
(BOW, LDA), unsupervised representation learning methods (Siamese CBOW,
Tweet2Vec) as well as supervised methods (CNN, BLSTM).
Constrained Low-Rank Learning Using Least Squares-Based Regularization
Authors: Ping Li, Jun Yu, Meng Wang, Luming Zhang, Deng Cai, Xuelong. Li
Date: 2016-11-15 00:00:00
Tasks: General Classification, Image Categorization, image-classification, Image Classification, Pose Estimation

Low-rank learning has attracted much attention recently due to its efficacy
in a rich variety of real-world tasks, e.g., subspace segmentation and image
categorization. Most low-rank methods are incapable of capturing
low-dimensional subspace for supervised learning tasks, e.g., classification
and regression. This paper aims to learn both the discriminant low-rank
representation (LRR) and the robust projecting subspace in a supervised manner.
To achieve this goal, we cast the problem into a constrained rank minimization
framework by adopting the least squares regularization. Naturally, the data
label structure tends to resemble that of the corresponding low-dimensional
representation, which is derived from the robust subspace projection of clean
data by low-rank learning. Moreover, the low-dimensional representation of
original data can be paired with some informative structure by imposing an
appropriate constraint, e.g., Laplacian regularizer. Therefore, we propose a
novel constrained LRR method. The objective function is formulated as a
constrained nuclear norm minimization problem, which can be solved by the
inexact augmented Lagrange multiplier algorithm. Extensive experiments on image
classification, human pose estimation, and robust face recovery have confirmed
the superiority of our method.
Character-level Convolutional Network for Text Classification Applied to Chinese Corpus
Authors: Wei-Jie Huang, Jun Wang
Date: 2016-11-14 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification

This article provides an interesting exploration of character-level
convolutional neural network solving Chinese corpus text classification
problem. We constructed a large-scale Chinese language dataset, and the result
shows that character-level convolutional neural network works better on Chinese
corpus than its corresponding pinyin format dataset. This is the first time
that character-level convolutional neural network applied to text
classification problem.
Scale-constrained Unsupervised Evaluation Method for Multi-scale Image Segmentation
Authors: Yuhang Lu, Youchuan Wan, Gang Li
Date: 2016-11-15 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Unsupervised evaluation of segmentation quality is a crucial step in image
segmentation applications. Previous unsupervised evaluation methods usually
lacked the adaptability to multi-scale segmentation. A scale-constrained
evaluation method that evaluates segmentation quality according to the
specified target scale is proposed in this paper. First, regional saliency and
merging cost are employed to describe intra-region homogeneity and inter-region
heterogeneity, respectively. Subsequently, both of them are standardized into
equivalent spectral distances of a predefined region. Finally, by analyzing the
relationship between image characteristics and segmentation quality, we
establish the evaluation model. Experimental results show that the proposed
method outperforms four commonly used unsupervised methods in multi-scale
evaluation tasks.
The Role of Word Length in Semantic Topology
Authors: Francesco Fumarola
Date: 2016-11-15 00:00:00
Tasks: Retrieval

A topological argument is presented concering the structure of semantic
space, based on the negative correlation between polysemy and word length. The
resulting graph structure is applied to the modeling of free-recall
experiments, resulting in predictions on the comparative values of recall
probabilities. Associative recall is found to favor longer words whereas
sequential recall is found to favor shorter words. Data from the PEERS
experiments of Lohnas et al. (2015) and Healey and Kahana (2016) confirm both
predictons, with correlation coefficients $r_{seq}= -0.17$ and $r_{ass}=
+0.17$. The argument is then applied to predicting global properties of list
recall, which leads to a novel explanation for the word-length effect based on
the optimization of retrieval strategies.
Multilinear Low-Rank Tensors on Graphs & Applications
Authors: Nauman Shahid, Francesco Grassi, Pierre Vandergheynst
Date: 2016-11-15 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

We propose a new framework for the analysis of low-rank tensors which lies at
the intersection of spectral graph theory and signal processing. As a first
step, we present a new graph based low-rank decomposition which approximates
the classical low-rank SVD for matrices and multi-linear SVD for tensors. Then,
building on this novel decomposition we construct a general class of convex
optimization problems for approximately solving low-rank tensor inverse
problems, such as tensor Robust PCA. The whole framework is named as
'Multilinear Low-rank tensors on Graphs (MLRTG)'. Our theoretical analysis
shows: 1) MLRTG stands on the notion of approximate stationarity of
multi-dimensional signals on graphs and 2) the approximation error depends on
the eigen gaps of the graphs. We demonstrate applications for a wide variety of
4 artificial and 12 real tensor datasets, such as EEG, FMRI, BCI, surveillance
videos and hyperspectral images. Generalization of the tensor concepts to
non-euclidean domain, orders of magnitude speed-up, low-memory requirement and
significantly enhanced performance at low SNR are the key aspects of our
framework.
Safe Exploration in Finite Markov Decision Processes with Gaussian Processes
Authors: Matteo Turchetta, Felix Berkenkamp, Andreas Krause
Date: 2016-06-15 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Gaussian Processes, Reinforcement Learning, Safe Exploration

In classical reinforcement learning, when exploring an environment, agents
accept arbitrary short term loss for long term gain. This is infeasible for
safety critical applications, such as robotics, where even a single unsafe
action may cause system failure. In this paper, we address the problem of
safely exploring finite Markov decision processes (MDP). We define safety in
terms of an, a priori unknown, safety constraint that depends on states and
actions. We aim to explore the MDP under this constraint, assuming that the
unknown function satisfies regularity conditions expressed via a Gaussian
process prior. We develop a novel algorithm for this task and prove that it is
able to completely explore the safely reachable part of the MDP without
violating the safety constraint. To achieve this, it cautiously explores safe
states and actions in order to gain statistical confidence about the safety of
unvisited state-action pairs from noisy observations collected while navigating
the environment. Moreover, the algorithm explicitly considers reachability when
exploring the MDP, ensuring that it does not get stuck in any state with no
safe way out. We demonstrate our method on digital terrain models for the task
of exploring an unknown map with a rover.
The Power of Normalization: Faster Evasion of Saddle Points
Authors: Kfir. Y. Levy
Date: 2016-11-15 00:00:00
Tasks: Tensor Decomposition

A commonly used heuristic in non-convex optimization is Normalized Gradient
Descent (NGD) - a variant of gradient descent in which only the direction of
the gradient is taken into account and its magnitude ignored. We analyze this
heuristic and show that with carefully chosen parameters and noise injection,
this method can provably evade saddle points. We establish the convergence of
NGD to a local minimum, and demonstrate rates which improve upon the fastest
known first order algorithm due to Ge e al. (2015).
  The effectiveness of our method is demonstrated via an application to the
problem of online tensor decomposition; a task for which saddle point evasion
is known to result in convergence to global minima.