pwc-archive/papers-with-abstracts huggingface dataset:
Signed Support Recovery for Single Index Models in High-Dimensions
Authors: Matey Neykov, Qian Lin, Jun S. Liu
Date: 2015-11-07 00:00:00
Tasks: Vocal Bursts Intensity Prediction

In this paper we study the support recovery problem for single index models
$Y=f(\boldsymbol{X}^{\intercal} \boldsymbol{\beta},\varepsilon)$, where $f$ is
an unknown link function, $\boldsymbol{X}\sim N_p(0,\mathbb{I}_{p})$ and
$\boldsymbol{\beta}$ is an $s$-sparse unit vector such that
$\boldsymbol{\beta}_{i}\in \{\pm\frac{1}{\sqrt{s}},0\}$. In particular, we look
into the performance of two computationally inexpensive algorithms: (a) the
diagonal thresholding sliced inverse regression (DT-SIR) introduced by Lin et
al. (2015); and (b) a semi-definite programming (SDP) approach inspired by
Amini & Wainwright (2008). When $s=O(p^{1-\delta})$ for some $\delta>0$, we
demonstrate that both procedures can succeed in recovering the support of
$\boldsymbol{\beta}$ as long as the rescaled sample size
$\kappa=\frac{n}{s\log(p-s)}$ is larger than a certain critical threshold. On
the other hand, when $\kappa$ is smaller than a critical value, any algorithm
fails to recover the support with probability at least $\frac{1}{2}$
asymptotically. In other words, we demonstrate that both DT-SIR and the SDP
approach are optimal (up to a scalar) for recovering the support of
$\boldsymbol{\beta}$ in terms of sample size. We provide extensive simulations,
as well as a real dataset application to help verify our theoretical
observations.
Link Prediction via Matrix Completion
Authors: Ratha Pech, Dong Hao, Liming Pan, Hong Cheng, Tao Zhou
Date: 2016-06-22 00:00:00
Tasks: Link Prediction, Matrix Completion, Prediction

Inspired by practical importance of social networks, economic networks,
biological networks and so on, studies on large and complex networks have
attracted a surge of attentions in the recent years. Link prediction is a
fundamental issue to understand the mechanisms by which new links are added to
the networks. We introduce the method of robust principal component analysis
(robust PCA) into link prediction, and estimate the missing entries of the
adjacency matrix. On one hand, our algorithm is based on the sparsity and low
rank property of the matrix, on the other hand, it also performs very well when
the network is dense. This is because a relatively dense real network is also
sparse in comparison to the complete graph. According to extensive experiments
on real networks from disparate fields, when the target network is connected
and sufficiently dense, whatever it is weighted or unweighted, our method is
demonstrated to be very effective and with prediction accuracy being
considerably improved comparing with many state-of-the-art algorithms.
L1-Regularized Least Squares for Support Recovery of High Dimensional Single Index Models with Gaussian Designs
Authors: Matey Neykov, Jun S. Liu, Tianxi Cai
Date: 2015-11-25 00:00:00

It is known that for a certain class of single index models (SIMs) $Y =
f(\boldsymbol{X}_{p \times 1}^\intercal\boldsymbol{\beta}_0, \varepsilon)$,
support recovery is impossible when $\boldsymbol{X} \sim \mathcal{N}(0,
\mathbb{I}_{p \times p})$ and a model complexity adjusted sample size is below
a critical threshold. Recently, optimal algorithms based on Sliced Inverse
Regression (SIR) were suggested. These algorithms work provably under the
assumption that the design $\boldsymbol{X}$ comes from an i.i.d. Gaussian
distribution. In the present paper we analyze algorithms based on covariance
screening and least squares with $L_1$ penalization (i.e. LASSO) and
demonstrate that they can also enjoy optimal (up to a scalar) rescaled sample
size in terms of support recovery, albeit under slightly different assumptions
on $f$ and $\varepsilon$ compared to the SIR based algorithms. Furthermore, we
show more generally, that LASSO succeeds in recovering the signed support of
$\boldsymbol{\beta}_0$ if $\boldsymbol{X} \sim \mathcal{N}(0,
\boldsymbol{\Sigma})$, and the covariance $\boldsymbol{\Sigma}$ satisfies the
irrepresentable condition. Our work extends existing results on the support
recovery of LASSO for the linear model, to a more general class of SIMs.
3D Display Calibration by Visual Pattern Analysis
Authors: Hyoseok Hwang, Hyun Sung Chang, Dongkyung Nam, In So Kweon
Date: 2016-06-23 00:00:00

Nearly all 3D displays need calibration for correct rendering. More often
than not, the optical elements in a 3D display are misaligned from the designed
parameter setting. As a result, 3D magic does not perform well as intended. The
observed images tend to get distorted. In this paper, we propose a novel
display calibration method to fix the situation. In our method, a pattern image
is displayed on the panel and a camera takes its pictures twice at different
positions. Then, based on a quantitative model, we extract all display
parameters (i.e., pitch, slanted angle, gap or thickness, offset) from the
observed patterns in the captured images. For high accuracy and robustness, our
method analyzes the patterns mostly in frequency domain. We conduct two types
of experiments for validation; one with optical simulation for quantitative
results and the other with real-life displays for qualitative assessment.
Experimental results demonstrate that our method is quite accurate, about a
half order of magnitude higher than prior work; is efficient, spending less
than 2 s for computation; and is robust to noise, working well in the SNR
regime as low as 6 dB.
Interpretable Machine Learning Models for the Digital Clock Drawing Test
Authors: William Souillard-Mandar, Randall Davis, Cynthia Rudin, Rhoda Au, Dana Penney
Date: 2016-06-23 00:00:00
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning

The Clock Drawing Test (CDT) is a rapid, inexpensive, and popular
neuropsychological screening tool for cognitive conditions. The Digital Clock
Drawing Test (dCDT) uses novel software to analyze data from a digitizing
ballpoint pen that reports its position with considerable spatial and temporal
precision, making possible the analysis of both the drawing process and final
product. We developed methodology to analyze pen stroke data from these
drawings, and computed a large collection of features which were then analyzed
with a variety of machine learning techniques. The resulting scoring systems
were designed to be more accurate than the systems currently used by
clinicians, but just as interpretable and easy to use. The systems also allow
us to quantify the tradeoff between accuracy and interpretability. We created
automated versions of the CDT scoring systems currently used by clinicians,
allowing us to benchmark our models, which indicated that our machine learning
models substantially outperformed the existing scoring systems.
A Unified Theory of Confidence Regions and Testing for High Dimensional Estimating Equations
Authors: Matey Neykov, Yang Ning, Jun S. Liu, Han Liu
Date: 2015-10-30 00:00:00
Tasks: compressed sensing, valid

We propose a new inferential framework for constructing confidence regions
and testing hypotheses in statistical models specified by a system of high
dimensional estimating equations. We construct an influence function by
projecting the fitted estimating equations to a sparse direction obtained by
solving a large-scale linear program. Our main theoretical contribution is to
establish a unified Z-estimation theory of confidence regions for high
dimensional problems.
  Different from existing methods, all of which require the specification of
the likelihood or pseudo-likelihood, our framework is likelihood-free. As a
result, our approach provides valid inference for a broad class of high
dimensional constrained estimating equation problems, which are not covered by
existing methods.
  Such examples include, noisy compressed sensing, instrumental variable
regression, undirected graphical models, discriminant analysis and vector
autoregressive models. We present detailed theoretical results for all these
examples. Finally, we conduct thorough numerical simulations, and a real
dataset analysis to back up the developed theoretical results.
E-commerce in Your Inbox: Product Recommendations at Scale
Authors: Mihajlo Grbovic, Vladan Radosavljevic, Nemanja Djuric, Narayan Bhamidipati, Jaikit Savla, Varun Bhagwan, Doug Sharp
Date: 2016-06-23 00:00:00

In recent years online advertising has become increasingly ubiquitous and
effective. Advertisements shown to visitors fund sites and apps that publish
digital content, manage social networks, and operate e-mail services. Given
such large variety of internet resources, determining an appropriate type of
advertising for a given platform has become critical to financial success.
Native advertisements, namely ads that are similar in look and feel to content,
have had great success in news and social feeds. However, to date there has not
been a winning formula for ads in e-mail clients. In this paper we describe a
system that leverages user purchase history determined from e-mail receipts to
deliver highly personalized product ads to Yahoo Mail users. We propose to use
a novel neural language-based algorithm specifically tailored for delivering
effective product recommendations, which was evaluated against baselines that
included showing popular products and products predicted based on
co-occurrence. We conducted rigorous offline testing using a large-scale
product purchase data set, covering purchases of more than 29 million users
from 172 e-commerce websites. Ads in the form of product recommendations were
successfully tested on online traffic, where we observed a steady 9% lift in
click-through rates over other ad formats in mail, as well as comparable lift
in conversion rates. Following successful tests, the system was launched into
production during the holiday season of 2014.
Fast robustness quantification with variational Bayes
Authors: Ryan Giordano, Tamara Broderick, Rachael Meager, Jonathan Huggins, Michael Jordan
Date: 2016-06-23 00:00:00

Bayesian hierarchical models are increasing popular in economics. When using
hierarchical models, it is useful not only to calculate posterior expectations,
but also to measure the robustness of these expectations to reasonable
alternative prior choices. We use variational Bayes and linear response methods
to provide fast, accurate posterior means and robustness measures with an
application to measuring the effectiveness of microcredit in the developing
world.
An Approach to Stable Gradient Descent Adaptation of Higher-Order Neural Units
Authors: Ivo Bukovsky, Noriyasu Homma
Date: 2016-06-23 00:00:00
Tasks: Polynomial Neural Networks

Stability evaluation of a weight-update system of higher-order neural units
(HONUs) with polynomial aggregation of neural inputs (also known as classes of
polynomial neural networks) for adaptation of both feedforward and recurrent
HONUs by a gradient descent method is introduced. An essential core of the
approach is based on spectral radius of a weight-update system, and it allows
stability monitoring and its maintenance at every adaptation step individually.
Assuring stability of the weight-update system (at every single adaptation
step) naturally results in adaptation stability of the whole neural
architecture that adapts to target data. As an aside, the used approach
highlights the fact that the weight optimization of HONU is a linear problem,
so the proposed approach can be generally extended to any neural architecture
that is linear in its adaptable parameters.
Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models
Authors: Minh-Thang Luong, Christopher D. Manning
Date: 2016-04-04 00:00:00
Proceeding: ACL 2016 8
Tasks: Machine Translation, NMT, Translation

Nearly all previous work on neural machine translation (NMT) has used quite
restricted vocabularies, perhaps with a subsequent method to patch in unknown
words. This paper presents a novel word-character solution to achieving open
vocabulary NMT. We build hybrid systems that translate mostly at the word level
and consult the character components for rare words. Our character-level
recurrent neural networks compute source word representations and recover
unknown target words when needed. The twofold advantage of such a hybrid
approach is that it is much faster and easier to train than character-based
ones; at the same time, it never produces unknown words as in the case of
word-based models. On the WMT'15 English to Czech translation task, this hybrid
approach offers an addition boost of +2.1-11.4 BLEU points over models that
already handle unknown words. Our best system achieves a new state-of-the-art
result with 20.7 BLEU score. We demonstrate that our character models can
successfully learn to not only generate well-formed words for Czech, a
highly-inflected language with a very complex vocabulary, but also build
correct representations for English source words.
Query Expansion with Locally-Trained Word Embeddings
Authors: Fernando Diaz, Bhaskar Mitra, Nick Craswell
Date: 2016-05-25 00:00:00
Proceeding: ACL 2016 8
Tasks: Ad-Hoc Information Retrieval, BIG-bench Machine Learning, Information Retrieval, Retrieval, Word Embeddings

Continuous space word embeddings have received a great deal of attention in
the natural language processing and machine learning communities for their
ability to model term similarity and other relationships. We study the use of
term relatedness in the context of query expansion for ad hoc information
retrieval. We demonstrate that word embeddings such as word2vec and GloVe, when
trained globally, underperform corpus and query specific embeddings for
retrieval tasks. These results suggest that other tasks benefiting from global
embeddings may also benefit from local embeddings.
Preliminaries of a Space Situational Awareness Ontology
Authors: Robert John Rovetto, T. S. Kelso
Date: 2016-06-02 00:00:00

Space situational awareness (SSA) is vital for international safety and
security, and the future of space travel. By improving SSA data-sharing we
improve global SSA. Computational ontology may provide one means toward that
goal. This paper develops the ontology of the SSA domain and takes steps in the
creation of the space situational awareness ontology. Ontology objectives,
requirements and desiderata are outlined; and both the SSA domain and the
discipline of ontology are described. The purposes of the ontology include:
exploring the potential for ontology development and engineering to (i)
represent SSA data, general domain knowledge, objects and relationships (ii)
annotate and express the meaning of that data, and (iii) foster SSA
data-exchange and integration among SSA actors, orbital debris databases, space
object catalogs and other SSA data repositories. By improving SSA via data- and
knowledge-sharing, we can (iv) expand our scientific knowledge of the space
environment, (v) advance our capacity for planetary defense from near-Earth
objects, and (vi) ensure the future of safe space flight for generations to
come.
Automated Extraction of Number of Subjects in Randomised Controlled Trials
Authors: Abeed Sarker
Date: 2016-06-22 00:00:00
Tasks: Articles, General Classification, Question Answering

We present a simple approach for automatically extracting the number of
subjects involved in randomised controlled trials (RCT). Our approach first
applies a set of rule-based techniques to extract candidate study sizes from
the abstracts of the articles. Supervised classification is then performed over
the candidates with support vector machines, using a small set of lexical,
structural, and contextual features. With only a small annotated training set
of 201 RCTs, we obtained an accuracy of 88\%. We believe that this system will
aid complex medical text processing tasks such as summarisation and question
answering.
Joint Learning of Sentence Embeddings for Relevance and Entailment
Authors: Petr Baudis, Silvestr Stanko, Jan Sedivy
Date: 2016-05-16 00:00:00
Proceeding: WS 2016 8
Tasks: Decision Making, Information Retrieval, Multiple-choice, Natural Language Inference, Reading Comprehension, Retrieval, Sentence, Sentence Embeddings

We consider the problem of Recognizing Textual Entailment within an
Information Retrieval context, where we must simultaneously determine the
relevancy as well as degree of entailment for individual pieces of evidence to
determine a yes/no answer to a binary natural language question.
  We compare several variants of neural networks for sentence embeddings in a
setting of decision-making based on evidence of varying relevance. We propose a
basic model to integrate evidence for entailment, show that joint training of
the sentence embeddings to model relevance and entailment is feasible even with
no explicit per-evidence supervision, and show the importance of evaluating
strong baselines. We also demonstrate the benefit of carrying over text
comprehension model trained on an unrelated task for our small datasets.
  Our research is motivated primarily by a new open dataset we introduce,
consisting of binary questions and news-based evidence snippets. We also apply
the proposed relevance-entailment model on a similar task of ranking
multiple-choice test answers, evaluating it on a preliminary dataset of school
test questions as well as the standard MCTest dataset, where we improve the
neural model state-of-art.
Explainable Restricted Boltzmann Machines for Collaborative Filtering
Authors: Behnoush Abdollahi, Olfa Nasraoui
Date: 2016-06-22 00:00:00
Tasks: Collaborative Filtering, Explanation Generation, Recommendation Systems

Most accurate recommender systems are black-box models, hiding the reasoning
behind their recommendations. Yet explanations have been shown to increase the
user's trust in the system in addition to providing other benefits such as
scrutability, meaning the ability to verify the validity of recommendations.
This gap between accuracy and transparency or explainability has generated an
interest in automated explanation generation methods. Restricted Boltzmann
Machines (RBM) are accurate models for CF that also lack interpretability. In
this paper, we focus on RBM based collaborative filtering recommendations, and
further assume the absence of any additional data source, such as item content
or user attributes. We thus propose a new Explainable RBM technique that
computes the top-n recommendation list from items that are explainable.
Experimental results show that our method is effective in generating accurate
and explainable recommendations.
Visualizing Dynamics: from t-SNE to SEMI-MDPs
Authors: Nir Ben Zrihem, Tom Zahavy, Shie Mannor
Date: 2016-06-22 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning

Deep Reinforcement Learning (DRL) is a trending field of research, showing
great promise in many challenging problems such as playing Atari, solving Go
and controlling robots. While DRL agents perform well in practice we are still
missing the tools to analayze their performance and visualize the temporal
abstractions that they learn. In this paper, we present a novel method that
automatically discovers an internal Semi Markov Decision Process (SMDP) model
in the Deep Q Network's (DQN) learned representation. We suggest a novel
visualization method that represents the SMDP model by a directed graph and
visualize it above a t-SNE map. We show how can we interpret the agent's policy
and give evidence for the hierarchical state aggregation that DQNs are learning
automatically. Our algorithm is fully automatic, does not require any domain
specific knowledge and is evaluated by a novel likelihood based evaluation
criteria.
Walk and Learn: Facial Attribute Representation Learning from Egocentric Video and Contextual Data
Authors: Jing Wang, Yu Cheng, Rogerio Schmidt Feris
Date: 2016-04-21 00:00:00
Proceeding: CVPR 2016 6
Tasks: Attribute, Facial Attribute Classification, Representation Learning

The way people look in terms of facial attributes (ethnicity, hair color,
facial hair, etc.) and the clothes or accessories they wear (sunglasses, hat,
hoodies, etc.) is highly dependent on geo-location and weather condition,
respectively. This work explores, for the first time, the use of this
contextual information, as people with wearable cameras walk across different
neighborhoods of a city, in order to learn a rich feature representation for
facial attribute classification, without the costly manual annotation required
by previous methods. By tracking the faces of casual walkers on more than 40
hours of egocentric video, we are able to cover tens of thousands of different
identities and automatically extract nearly 5 million pairs of images connected
by or from different face tracks, along with their weather and location
context, under pose and lighting variations. These image pairs are then fed
into a deep network that preserves similarity of images connected by the same
track, in order to capture identity-related attribute features, and optimizes
for location and weather prediction to capture additional facial attribute
features. Finally, the network is fine-tuned with manually annotated samples.
We perform an extensive experimental analysis on wearable data and two standard
benchmark datasets based on web images (LFWA and CelebA). Our method
outperforms by a large margin a network trained from scratch. Moreover, even
without using manually annotated identity labels for pre-training as in
previous methods, our approach achieves results that are better than the state
of the art.
Finding Proofs in Tarskian Geometry
Authors: Michael Beeson, Larry Wos
Date: 2016-06-22 00:00:00

We report on a project to use a theorem prover to find proofs of the theorems
in Tarskian geometry. These theorems start with fundamental properties of
betweenness, proceed through the derivations of several famous theorems due to
Gupta and end with the derivation from Tarski's axioms of Hilbert's 1899 axioms
for geometry. They include the four challenge problems left unsolved by Quaife,
who two decades ago found some \Otter proofs in Tarskian geometry (solving
challenges issued in Wos's 1998 book). There are 212 theorems in this
collection. We were able to find \Otter proofs of all these theorems. We
developed a methodology for the automated preparation and checking of the input
files for those theorems, to ensure that no human error has corrupted the
formal development of an entire theory as embodied in two hundred input files
and proofs. We distinguish between proofs that were found completely
mechanically (without reference to the steps of a book proof) and proofs that
were constructed by some technique that involved a human knowing the steps of a
book proof. Proofs of length 40--100, roughly speaking, are difficult exercises
for a human, and proofs of 100-250 steps belong in a Ph.D. thesis or
publication. 29 of the proofs in our collection are longer than 40 steps, and
ten are longer than 90 steps. We were able to derive completely mechanically
all but 26 of the 183 theorems that have "short" proofs (40 or fewer deduction
steps). We found proofs of the rest, as well as the 29 "hard" theorems, using a
method that requires consulting the book proof at the outset. Our "subformula
strategy" enabled us to prove four of the 29 hard theorems completely
mechanically. These are Ph.D. level proofs, of length up to 108.