pwc-archive/papers-with-abstracts huggingface dataset:
Exhaustive search for sparse variable selection in linear regression
Authors: Yasuhiko Igarashi, Hikaru Takenaka, Yoshinori Nakanishi-Ohno, Makoto Uemura, Shiro Ikeda, Masato Okada
Date: 2017-07-07 00:00:00
Tasks: Astronomy, regression, Variable Selection

We propose a K-sparse exhaustive search (ES-K) method and a K-sparse
approximate exhaustive search method (AES-K) for selecting variables in linear
regression. With these methods, K-sparse combinations of variables are tested
exhaustively assuming that the optimal combination of explanatory variables is
K-sparse. By collecting the results of exhaustively computing ES-K, various
approximate methods for selecting sparse variables can be summarized as density
of states. With this density of states, we can compare different methods for
selecting sparse variables such as relaxation and sampling. For large problems
where the combinatorial explosion of explanatory variables is crucial, the
AES-K method enables density of states to be effectively reconstructed by using
the replica-exchange Monte Carlo method and the multiple histogram method.
Applying the ES-K and AES-K methods to type Ia supernova data, we confirmed the
conventional understanding in astronomy when an appropriate K is given
beforehand. However, we found the difficulty to determine K from the data.
Using virtual measurement and analysis, we argue that this is caused by data
shortage.
Learning to Optimize via Information-Directed Sampling
Authors: Daniel Russo, Benjamin Van Roy
Date: 2014-03-21 00:00:00
Proceeding: NeurIPS 2014 12

We propose information-directed sampling -- a new approach to online
optimization problems in which a decision-maker must balance between
exploration and exploitation while learning from partial feedback. Each action
is sampled in a manner that minimizes the ratio between squared expected
single-period regret and a measure of information gain: the mutual information
between the optimal action and the next observation. We establish an expected
regret bound for information-directed sampling that applies across a very
general class of models and scales with the entropy of the optimal action
distribution. We illustrate through simple analytic examples how
information-directed sampling accounts for kinds of information that
alternative approaches do not adequately address and that this can lead to
dramatic performance gains. For the widely studied Bernoulli, Gaussian, and
linear bandit problems, we demonstrate state-of-the-art simulation performance.
Networked Fairness in Cake Cutting
Authors: Xiaohui Bei, Youming Qiao, Shengyu Zhang
Date: 2017-07-07 00:00:00
Tasks: Fairness

We introduce a graphical framework for fair division in cake cutting, where
comparisons between agents are limited by an underlying network structure. We
generalize the classical fairness notions of envy-freeness and proportionality
to this graphical setting. Given a simple undirected graph G, an allocation is
envy-free on G if no agent envies any of her neighbor's share, and is
proportional on G if every agent values her own share no less than the average
among her neighbors, with respect to her own measure. These generalizations
open new research directions in developing simple and efficient algorithms that
can produce fair allocations under specific graph structures.
  On the algorithmic frontier, we first propose a moving-knife algorithm that
outputs an envy-free allocation on trees. The algorithm is significantly
simpler than the discrete and bounded envy-free algorithm recently designed by
Aziz and Mackenzie for complete graphs. Next, we give a discrete and bounded
algorithm for computing a proportional allocation on descendant graphs, a class
of graphs by taking a rooted tree and connecting all its ancestor-descendant
pairs.
LoopInvGen: A Loop Invariant Generator based on Precondition Inference
Authors: Saswat Padhi, Rahul Sharma, Todd Millstein
Date: 2017-07-07 00:00:00
Tasks: Program Synthesis

We describe the LoopInvGen tool for generating loop invariants that can provably guarantee correctness of a program with respect to a given specification. LoopInvGen is an efficient implementation of the inference technique originally proposed in our earlier work on PIE (https://doi.org/10.1145/2908080.2908099). In contrast to existing techniques, LoopInvGen is not restricted to a fixed set of features -- atomic predicates that are composed together to build complex loop invariants. Instead, we start with no initial features, and use program synthesis techniques to grow the set on demand. This not only enables a less onerous and more expressive approach, but also appears to be significantly faster than the existing tools over the SyGuS-COMP 2018 benchmarks from the INV track.
TasselNet: Counting maize tassels in the wild via local counts regression network
Authors: Hao Lu, Zhiguo Cao, Yang Xiao, Bohan Zhuang, Chunhua Shen
Date: 2017-07-07 00:00:00
Tasks: Plant Phenotyping, regression

Accurately counting maize tassels is important for monitoring the growth
status of maize plants. This tedious task, however, is still mainly done by
manual efforts. In the context of modern plant phenotyping, automating this
task is required to meet the need of large-scale analysis of genotype and
phenotype. In recent years, computer vision technologies have experienced a
significant breakthrough due to the emergence of large-scale datasets and
increased computational resources. Naturally image-based approaches have also
received much attention in plant-related studies. Yet a fact is that most
image-based systems for plant phenotyping are deployed under controlled
laboratory environment. When transferring the application scenario to
unconstrained in-field conditions, intrinsic and extrinsic variations in the
wild pose great challenges for accurate counting of maize tassels, which goes
beyond the ability of conventional image processing techniques. This calls for
further robust computer vision approaches to address in-field variations. This
paper studies the in-field counting problem of maize tassels. To our knowledge,
this is the first time that a plant-related counting problem is considered
using computer vision technologies under unconstrained field-based environment.
Option Pricing and Hedging for Discrete Time Autoregressive Hidden Markov Model
Authors: Massimo Caccia, Bruno Rémillard
Date: 2017-07-07 00:00:00
Tasks: Time Series, Time Series Analysis

In this paper we solve the discrete time mean-variance hedging problem when
asset returns follow a multivariate autoregressive hidden Markov model. Time
dependent volatility and serial dependence are well established properties of
financial time series and our model covers both. To illustrate the relevance of
our proposed methodology, we first compare the proposed model with the
well-known hidden Markov model via likelihood ratio tests and a novel
goodness-of-fit test on the S\&P 500 daily returns. Secondly, we present
out-of-sample hedging results on S\&P 500 vanilla options as well as a trading
strategy based on theoretical prices, which we compare to simpler models
including the classical Black-Scholes delta-hedging approach.
Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
Authors: Alex Beutel, Jilin Chen, Zhe Zhao, Ed H. Chi
Date: 2017-07-01 00:00:00
Tasks: Attribute, Fairness, Recommendation Systems

How can we learn a classifier that is "fair" for a protected or sensitive
group, when we do not know if the input to the classifier belongs to the
protected group? How can we train such a classifier when data on the protected
group is difficult to attain? In many settings, finding out the sensitive input
attribute can be prohibitively expensive even during model training, and
sometimes impossible during model serving. For example, in recommender systems,
if we want to predict if a user will click on a given recommendation, we often
do not know many attributes of the user, e.g., race or age, and many attributes
of the content are hard to determine, e.g., the language or topic. Thus, it is
not feasible to use a different classifier calibrated based on knowledge of the
sensitive attribute.
  Here, we use an adversarial training procedure to remove information about
the sensitive attribute from the latent representation learned by a neural
network. In particular, we study how the choice of data for the adversarial
training effects the resulting fairness properties. We find two interesting
results: a small amount of data is needed to train these adversarial models,
and the data distribution empirically drives the adversary's notion of
fairness.
Multi-view (Joint) Probability Linear Discrimination Analysis for Multi-view Feature Verification
Authors: Ziqiang Shi, Liu Liu, Mengjiao Wang, Rujie Liu
Date: 2017-04-20 00:00:00
Tasks: Decision Making

Multi-view feature has been proved to be very effective in many multimedia
applications. However, the current back-end classifiers cannot make full use of
such features. In this paper, we propose a method to model the multi-faceted
information in the multi-view features explicitly and jointly. In our approach,
the feature was modeled as a result derived by a generative multi-view
(joint\footnotemark[1]) Probability Linear Discriminant Analysis (PLDA) model,
which contains multiple kinds of latent variables. The usual PLDA model only
considers one single label. However, in practical use, when using multi-task
learned network as feature extractor, the extracted feature are always attached
to several labels. This type of feature is called multi-view feature. With
multi-view (joint) PLDA, we are able to explicitly build a model that can
combine multiple heterogeneous information from the multi-view features. In
verification step, we calculated the likelihood to describe whether the two
features having consistent labels or not. This likelihood are used in the
following decision-making. Experiments have been conducted on large scale
verification task. On the public RSR2015 data corpus, the results showed that
our approach can achieve 0.02\% EER and 0.09\% EER for impostor wrong and
impostor correct cases respectively.
Online Rules for Control of False Discovery Rate and False Discovery Exceedance
Authors: Adel Javanmard, Andrea Montanari
Date: 2016-03-29 00:00:00
Tasks: Two-sample testing

Multiple hypothesis testing is a core problem in statistical inference and
arises in almost every scientific field. Given a set of null hypotheses
$\mathcal{H}(n) = (H_1,\dotsc, H_n)$, Benjamini and Hochberg introduced the
false discovery rate (FDR), which is the expected proportion of false positives
among rejected null hypotheses, and proposed a testing procedure that controls
FDR below a pre-assigned significance level. Nowadays FDR is the criterion of
choice for large scale multiple hypothesis testing. In this paper we consider
the problem of controlling FDR in an "online manner". Concretely, we consider
an ordered --possibly infinite-- sequence of null hypotheses $\mathcal{H} =
(H_1,H_2,H_3,\dots )$ where, at each step $i$, the statistician must decide
whether to reject hypothesis $H_i$ having access only to the previous
decisions. This model was introduced by Foster and Stine. We study a class of
"generalized alpha-investing" procedures and prove that any rule in this class
controls online FDR, provided $p$-values corresponding to true nulls are
independent from the other $p$-values. (Earlier work only established mFDR
control.) Next, we obtain conditions under which generalized alpha-investing
controls FDR in the presence of general $p$-values dependencies. Finally, we
develop a modified set of procedures that also allow to control the false
discovery exceedance (the tail of the proportion of false discoveries).
Numerical simulations and analytical results indicate that online procedures do
not incur a large loss in statistical power with respect to offline approaches,
such as Benjamini-Hochberg.
On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
Authors: Wenqi Li, Guotai Wang, Lucas Fidon, Sebastien Ourselin, M. Jorge Cardoso, Tom Vercauteren
Date: 2017-07-06 00:00:00
Tasks: 3D Medical Imaging Segmentation, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation, Transfer Learning, Volumetric Medical Image Segmentation

Deep convolutional neural networks are powerful tools for learning visual
representations from images. However, designing efficient deep architectures to
analyse volumetric medical images remains challenging. This work investigates
efficient and flexible elements of modern convolutional networks such as
dilated convolution and residual connection. With these essential building
blocks, we propose a high-resolution, compact convolutional network for
volumetric image segmentation. To illustrate its efficiency of learning 3D
representation from large-scale image data, the proposed network is validated
with the challenging task of parcellating 155 neuroanatomical structures from
brain MR images. Our experiments show that the proposed network architecture
compares favourably with state-of-the-art volumetric segmentation networks
while being an order of magnitude more compact. We consider the brain
parcellation task as a pretext task for volumetric image segmentation; our
trained network potentially provides a good starting point for transfer
learning. Additionally, we show the feasibility of voxel-level uncertainty
estimation using a sampling approximation through dropout.
Generalized Sparse Precision Matrix Selection for Fitting Multivariate Gaussian Random Fields to Large Data Sets
Authors: Sam Davanloo Tajbakhsh, Necdet Serhat Aybat, Enrique del Castillo
Date: 2016-05-11 00:00:00

We present a new method for estimating multivariate, second-order stationary
Gaussian Random Field (GRF) models based on the Sparse Precision matrix
Selection (SPS) algorithm, proposed by Davanloo et al. (2015) for estimating
scalar GRF models. Theoretical convergence rates for the estimated
between-response covariance matrix and for the estimated parameters of the
underlying spatial correlation function are established. Numerical tests using
simulated and real datasets validate our theoretical findings. Data
segmentation is used to handle large data sets.
Long-Term Memory Networks for Question Answering
Authors: Fenglong Ma, Radha Chitta, Saurabh Kataria, Jing Zhou, Palghat Ramesh, Tong Sun, Jing Gao
Date: 2017-07-06 00:00:00
Tasks: Question Answering

Question answering is an important and difficult task in the natural language
processing domain, because many basic natural language processing tasks can be
cast into a question answering task. Several deep neural network architectures
have been developed recently, which employ memory and inference components to
memorize and reason over text information, and generate answers to questions.
However, a major drawback of many such models is that they are capable of only
generating single-word answers. In addition, they require large amount of
training data to generate accurate answers. In this paper, we introduce the
Long-Term Memory Network (LTMN), which incorporates both an external memory
module and a Long Short-Term Memory (LSTM) module to comprehend the input data
and generate multi-word answers. The LTMN model can be trained end-to-end using
back-propagation and requires minimal supervision. We test our model on two
synthetic data sets (based on Facebook's bAbI data set) and the real-world
Stanford question answering data set, and show that it can achieve
state-of-the-art performance.
Robust Regression For Image Binarization Under Heavy Noises and Nonuniform Background
Authors: Garret Vo, Chiwoo Park
Date: 2016-09-26 00:00:00
Tasks: Binarization, Image Segmentation, Model Selection, regression, Semantic Segmentation

This paper presents a robust regression approach for image binarization under
significant background variations and observation noises. The work is motivated
by the need of identifying foreground regions in noisy microscopic image or
degraded document images, where significant background variation and severe
noise make an image binarization challenging. The proposed method first
estimates the background of an input image, subtracts the estimated background
from the input image, and apply a global thresholding to the subtracted outcome
for achieving a binary image of foregrounds. A robust regression approach was
proposed to estimate the background intensity surface with minimal effects of
foreground intensities and noises, and a global threshold selector was proposed
on the basis of a model selection criterion in a sparse regression. The
proposed approach was validated using 26 test images and the corresponding
ground truths, and the outcomes of the proposed work were compared with those
from nine existing image binarization methods. The approach was also combined
with three state-of-the-art morphological segmentation methods to show how the
proposed approach can improve their image segmentation outcomes.
Simple Classification using Binary Data
Authors: Deanna Needell, Rayan Saab, Tina Woolf
Date: 2017-07-06 00:00:00
Tasks: Classification, General Classification

Binary, or one-bit, representations of data arise naturally in many
applications, and are appealing in both hardware implementations and algorithm
design. In this work, we study the problem of data classification from binary
data and propose a framework with low computation and resource costs. We
illustrate the utility of the proposed approach through stylized and realistic
numerical experiments, and provide a theoretical analysis for a simple case. We
hope that our framework and analysis will serve as a foundation for studying
similar types of approaches.
High-Performance FPGA Implementation of Equivariant Adaptive Separation via Independence Algorithm for Independent Component Analysis
Authors: Mahdi Nazemi, Shahin Nazarian, Massoud Pedram
Date: 2017-07-06 00:00:00
Tasks: BIG-bench Machine Learning, Dimensionality Reduction

Independent Component Analysis (ICA) is a dimensionality reduction technique
that can boost efficiency of machine learning models that deal with probability
density functions, e.g. Bayesian neural networks. Algorithms that implement
adaptive ICA converge slower than their nonadaptive counterparts, however, they
are capable of tracking changes in underlying distributions of input features.
This intrinsically slow convergence of adaptive methods combined with existing
hardware implementations that operate at very low clock frequencies necessitate
fundamental improvements in both algorithm and hardware design. This paper
presents an algorithm that allows efficient hardware implementation of ICA.
Compared to previous work, our FPGA implementation of adaptive ICA improves
clock frequency by at least one order of magnitude and throughput by at least
two orders of magnitude. Our proposed algorithm is not limited to ICA and can
be used in various machine learning problems that use stochastic gradient
descent optimization.
Optimal choice: new machine learning problem and its solution
Authors: Marina Sapir
Date: 2017-06-26 00:00:00
Tasks: BIG-bench Machine Learning, Learning Theory

The task of learning to pick a single preferred example out a finite set of
examples, an "optimal choice problem", is a supervised machine learning problem
with complex, structured input. Problems of optimal choice emerge often in
various practical applications. We formalize the problem, show that it does not
satisfy the assumptions of statistical learning theory, yet it can be solved
efficiently in some cases. We propose two approaches to solve the problem. Both
of them reach good solutions on real life data from a signal processing
application.
Calibrated Fairness in Bandits
Authors: Yang Liu, Goran Radanovic, Christos Dimitrakakis, Debmalya Mandal, David C. Parkes
Date: 2017-07-06 00:00:00
Tasks: Decision Making, Fairness, Thompson Sampling

We study fairness within the stochastic, \emph{multi-armed bandit} (MAB)
decision making framework. We adapt the fairness framework of "treating similar
individuals similarly" to this setting. Here, an `individual' corresponds to an
arm and two arms are `similar' if they have a similar quality distribution.
First, we adopt a {\em smoothness constraint} that if two arms have a similar
quality distribution then the probability of selecting each arm should be
similar. In addition, we define the {\em fairness regret}, which corresponds to
the degree to which an algorithm is not calibrated, where perfect calibration
requires that the probability of selecting an arm is equal to the probability
with which the arm has the best quality realization. We show that a variation
on Thompson sampling satisfies smooth fairness for total variation distance,
and give an $\tilde{O}((kT)^{2/3})$ bound on fairness regret. This complements
prior work, which protects an on-average better arm from being less favored. We
also explain how to extend our algorithm to the dueling bandit setting.
Learning Semantic Part-Based Models from Google Images
Authors: Davide Modolo, Vittorio Ferrari
Date: 2016-09-11 00:00:00
Tasks: Object, object-detection, Object Detection

We propose a technique to train semantic part-based models of object classes
from Google Images. Our models encompass the appearance of parts and their
spatial arrangement on the object, specific to each viewpoint. We learn these
rich models by collecting training instances for both parts and objects, and
automatically connecting the two levels. Our framework works incrementally, by
learning from easy examples first, and then gradually adapting to harder ones.
A key benefit of this approach is that it requires no manual part location
annotations. We evaluate our models on the challenging PASCAL-Part dataset [1]
and show how their performance increases at every step of the learning, with
the final models more than doubling the performance of directly training from
images retrieved by querying for part names (from 12.9 to 27.2 AP). Moreover,
we show that our part models can help object detection performance by enriching
the R-CNN detector with parts.
Exploring Generalization in Deep Learning
Authors: Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, Nathan Srebro
Date: 2017-06-27 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Deep Learning

With a goal of understanding what drives generalization in deep networks, we
consider several recently suggested explanations, including norm-based control,
sharpness and robustness. We study how these measures can ensure
generalization, highlighting the importance of scale normalization, and making
a connection between sharpness and PAC-Bayes theory. We then investigate how
well the measures explain different observed phenomena.
Analogical Inference for Multi-Relational Embeddings
Authors: Hanxiao Liu, Yuexin Wu, Yiming Yang
Date: 2017-05-06 00:00:00
Proceeding: ICML 2017 8
Tasks: Knowledge Graphs, Link Prediction

Large-scale multi-relational embedding refers to the task of learning the
latent representations for entities and relations in large knowledge graphs. An
effective and scalable solution for this problem is crucial for the true
success of knowledge-based inference in a broad range of applications. This
paper proposes a novel framework for optimizing the latent representations with
respect to the \textit{analogical} properties of the embedded entities and
relations. By formulating the learning objective in a differentiable fashion,
our model enjoys both theoretical power and computational scalability, and
significantly outperformed a large number of representative baseline methods on
benchmark datasets. Furthermore, the model offers an elegant unification of
several well-known methods in multi-relational embedding, which can be proven
to be special instantiations of our framework.