pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Fly by Crashing
Authors: Dhiraj Gandhi, Lerrel Pinto, Abhinav Gupta
Date: 2017-04-19 00:00:00
Tasks: Navigate

How do you learn to navigate an Unmanned Aerial Vehicle (UAV) and avoid
obstacles? One approach is to use a small dataset collected by human experts:
however, high capacity learning algorithms tend to overfit when trained with
little data. An alternative is to use simulation. But the gap between
simulation and real world remains large especially for perception problems. The
reason most research avoids using large-scale real data is the fear of crashes!
In this paper, we propose to bite the bullet and collect a dataset of crashes
itself! We build a drone whose sole purpose is to crash into objects: it
samples naive trajectories and crashes into random objects. We crash our drone
11,500 times to create one of the biggest UAV crash dataset. This dataset
captures the different ways in which a UAV can crash. We use all this negative
flying data in conjunction with positive data sampled from the same
trajectories to learn a simple yet powerful policy for UAV navigation. We show
that this simple self-supervised model is quite effective in navigating the UAV
even in extremely cluttered environments with dynamic obstacles including
humans. For supplementary video see: https://youtu.be/u151hJaGKUo
Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks
Authors: Rajarshi Das, Manzil Zaheer, Siva Reddy, Andrew McCallum
Date: 2017-04-27 00:00:00
Proceeding: ACL 2017 7
Tasks: Question Answering

Existing question answering methods infer answers either from a knowledge
base or from raw text. While knowledge base (KB) methods are good at answering
compositional questions, their performance is often affected by the
incompleteness of the KB. Au contraire, web text contains millions of facts
that are absent in the KB, however in an unstructured form. {\it Universal
schema} can support reasoning on the union of both structured KBs and
unstructured text by aligning them in a common embedded space. In this paper we
extend universal schema to natural language question answering, employing
\emph{memory networks} to attend to the large body of facts in the combination
of text and KB. Our models can be trained in an end-to-end fashion on
question-answer pairs. Evaluation results on \spades fill-in-the-blank question
answering dataset show that exploiting universal schema for question answering
is better than using either a KB or text alone. This model also outperforms the
current state-of-the-art by 8.5 $F_1$ points.\footnote{Code and data available
in \url{https://rajarshd.github.io/TextKBQA}}
Large-scale Feature Selection of Risk Genetic Factors for Alzheimer's Disease via Distributed Group Lasso Regression
Authors: Qingyang Li, Dajiang Zhu, Jie Zhang, Derrek Paul Hibar, Neda Jahanshad, Yalin Wang, Jieping Ye, Paul M. Thompson, Jie Wang
Date: 2017-04-27 00:00:00
Tasks: feature selection

Genome-wide association studies (GWAS) have achieved great success in the
genetic study of Alzheimer's disease (AD). Collaborative imaging genetics
studies across different research institutions show the effectiveness of
detecting genetic risk factors. However, the high dimensionality of GWAS data
poses significant challenges in detecting risk SNPs for AD. Selecting relevant
features is crucial in predicting the response variable. In this study, we
propose a novel Distributed Feature Selection Framework (DFSF) to conduct the
large-scale imaging genetics studies across multiple institutions. To speed up
the learning process, we propose a family of distributed group Lasso screening
rules to identify irrelevant features and remove them from the optimization.
Then we select the relevant group features by performing the group Lasso
feature selection process in a sequence of parameters. Finally, we employ the
stability selection to rank the top risk SNPs that might help detect the early
stage of AD. To the best of our knowledge, this is the first distributed
feature selection model integrated with group Lasso feature selection as well
as detecting the risk genetic factors across multiple research institutions
system. Empirical studies are conducted on 809 subjects with 5.9 million SNPs
which are distributed across several individual institutions, demonstrating the
efficiency and effectiveness of the proposed method.
A New Type of Neurons for Machine Learning
Authors: Fenglei Fan, Wenxiang Cong, Ge Wang
Date: 2017-04-26 00:00:00
Tasks: BIG-bench Machine Learning, Vocal Bursts Type Prediction

In machine learning, the use of an artificial neural network is the
mainstream approach. Such a network consists of layers of neurons. These
neurons are of the same type characterized by the two features: (1) an inner
product of an input vector and a matching weighting vector of trainable
parameters and (2) a nonlinear excitation function. Here we investigate the
possibility of replacing the inner product with a quadratic function of the
input vector, thereby upgrading the 1st order neuron to the 2nd order neuron,
empowering individual neurons, and facilitating the optimization of neural
networks. Also, numerical examples are provided to illustrate the feasibility
and merits of the 2nd order neurons. Finally, further topics are discussed.
Identifying Similarities in Epileptic Patients for Drug Resistance Prediction
Authors: David Von Dollen
Date: 2017-04-26 00:00:00
Tasks: Clustering

Currently, approximately 30% of epileptic patients treated with antiepileptic
drugs (AEDs) remain resistant to treatment (known as refractory patients). This
project seeks to understand the underlying similarities in refractory patients
vs. other epileptic patients, identify features contributing to drug resistance
across underlying phenotypes for refractory patients, and develop predictive
models for drug resistance in epileptic patients. In this study, epileptic
patient data was examined to attempt to observe discernable similarities or
differences in refractory patients (case) and other non-refractory patients
(control) to map underlying mechanisms in causality. For the first part of the
study, unsupervised algorithms such as Kmeans, Spectral Clustering, and
Gaussian Mixture Models were used to examine patient features projected into a
lower dimensional space. Results from this study showed a high degree of
non-linearity in the underlying feature space. For the second part of this
study, classification algorithms such as Logistic Regression, Gradient Boosted
Decision Trees, and SVMs, were tested on the reduced-dimensionality features,
with accuracy results of 0.83(+/-0.3) testing using 7 fold cross validation.
Observations of test results indicate using a radial basis function kernel PCA
to reduce features ingested by a Gradient Boosted Decision Tree Ensemble lead
to gains in improved accuracy in mapping a binary decision to highly non-linear
features collected from epileptic patients.
Konzept für Bildanalysen in Hochdurchsatz-Systemen am Beispiel des Zebrabärblings
Authors: Rüdiger Alshut
Date: 2017-04-26 00:00:00
Tasks: General Classification

With image-based high-throughput experiments, new challenges arise in both,
the design of experiments and the automated analysis. To be able to handle the
massive number of single experiments and the corresponding amount of data, a
comprehensive concept for the design of experiments and a new evaluation method
is needed. This work proposes a new method for an optimized experiment layout
that enables the determination of parameters, adapted for the needs of
automated image analysis. Furthermore, a catalogue of new image analysis
modules, especially developed for zebrafish analysis, is presented. The
combination of both parts offers the user, usually a biologist, an approach for
high-throughput zebrafish image analysis, which enables the extraction of new
signals and optimizes the design of experiments. The result is a reduction of
data amount, redundant information and workload as well as classification
errors.
From Characters to Words to in Between: Do We Capture Morphology?
Authors: Clara Vania, Adam Lopez
Date: 2017-04-26 00:00:00
Proceeding: ACL 2017 7
Tasks: Language Modeling, Language Modelling

Words can be represented by composing the representations of subword units
such as word segments, characters, and/or character n-grams. While such
representations are effective and may capture the morphological regularities of
words, they have not been systematically compared, and it is not understood how
they interact with different morphological typologies. On a language modeling
task, we present experiments that systematically vary (1) the basic unit of
representation, (2) the composition of these representations, and (3) the
morphological typology of the language modeled. Our results extend previous
findings that character representations are effective across typologies, and we
find that a previously unstudied combination of character trigram
representations composed with bi-LSTMs outperforms most others. But we also
find room for improvement: none of the character-level models match the
predictive accuracy of a model with access to true morphological analyses, even
when learned from an order of magnitude more data.
The MacGyver Test - A Framework for Evaluating Machine Resourcefulness and Creative Problem Solving
Authors: Vasanth Sarathy, Matthias Scheutz
Date: 2017-04-26 00:00:00

Current measures of machine intelligence are either difficult to evaluate or
lack the ability to test a robot's problem-solving capacity in open worlds. We
propose a novel evaluation framework based on the formal notion of MacGyver
Test which provides a practical way for assessing the resilience and
resourcefulness of artificial agents.
SOFAR: large-scale association network learning
Authors: Yoshimasa Uematsu, Yingying Fan, Kun Chen, Jinchi Lv, Wei. Lin
Date: 2017-04-26 00:00:00

Many modern big data applications feature large scale in both numbers of
responses and predictors. Better statistical efficiency and scientific insights
can be enabled by understanding the large-scale response-predictor association
network structures via layers of sparse latent factors ranked by importance.
Yet sparsity and orthogonality have been two largely incompatible goals. To
accommodate both features, in this paper we suggest the method of sparse
orthogonal factor regression (SOFAR) via the sparse singular value
decomposition with orthogonality constrained optimization to learn the
underlying association networks, with broad applications to both unsupervised
and supervised learning tasks such as biclustering with sparse singular value
decomposition, sparse principal component analysis, sparse factor analysis, and
spare vector autoregression analysis. Exploiting the framework of
convexity-assisted nonconvex optimization, we derive nonasymptotic error bounds
for the suggested procedure characterizing the theoretical advantages. The
statistical guarantees are powered by an efficient SOFAR algorithm with
convergence property. Both computational and theoretical advantages of our
procedure are demonstrated with several simulation and real data examples.
On Using Active Learning and Self-Training when Mining Performance Discussions on Stack Overflow
Authors: Markus Borg, Iben Lennerstad, Rasmus Ros, Elizabeth Bjarnason
Date: 2017-04-26 00:00:00
Tasks: Active Learning, General Classification

Abundant data is the key to successful machine learning. However, supervised
learning requires annotated data that are often hard to obtain. In a
classification task with limited resources, Active Learning (AL) promises to
guide annotators to examples that bring the most value for a classifier. AL can
be successfully combined with self-training, i.e., extending a training set
with the unlabelled examples for which a classifier is the most certain. We
report our experiences on using AL in a systematic manner to train an SVM
classifier for Stack Overflow posts discussing performance of software
components. We show that the training examples deemed as the most valuable to
the classifier are also the most difficult for humans to annotate. Despite
carefully evolved annotation criteria, we report low inter-rater agreement, but
we also propose mitigation strategies. Finally, based on one annotator's work,
we show that self-training can improve the classification accuracy. We conclude
the paper by discussing implication for future text miners aspiring to use AL
and self-training.
Semantic Autoencoder for Zero-Shot Learning
Authors: Elyor Kodirov, Tao Xiang, Shaogang Gong
Date: 2017-04-26 00:00:00
Proceeding: CVPR 2017 7
Tasks: Attribute, Clustering, Decoder, Zero-Shot Learning

Existing zero-shot learning (ZSL) models typically learn a projection
function from a feature space to a semantic embedding space (e.g.~attribute
space). However, such a projection function is only concerned with predicting
the training seen class semantic representation (e.g.~attribute prediction) or
classification. When applied to test data, which in the context of ZSL contains
different (unseen) classes without training data, a ZSL model typically suffers
from the project domain shift problem. In this work, we present a novel
solution to ZSL based on learning a Semantic AutoEncoder (SAE). Taking the
encoder-decoder paradigm, an encoder aims to project a visual feature vector
into the semantic space as in the existing ZSL models. However, the decoder
exerts an additional constraint, that is, the projection/code must be able to
reconstruct the original visual feature. We show that with this additional
reconstruction constraint, the learned projection function from the seen
classes is able to generalise better to the new unseen classes. Importantly,
the encoder and decoder are linear and symmetric which enable us to develop an
extremely efficient learning algorithm. Extensive experiments on six benchmark
datasets demonstrate that the proposed SAE outperforms significantly the
existing ZSL models with the additional benefit of lower computational cost.
Furthermore, when the SAE is applied to supervised clustering problem, it also
beats the state-of-the-art.
Face Identification and Clustering
Authors: Atul Dhingra
Date: 2017-04-26 00:00:00
Tasks: Clustering, Face Identification, Face Recognition

In this thesis, we study two problems based on clustering algorithms. In the
first problem, we study the role of visual attributes using an agglomerative
clustering algorithm to whittle down the search area where the number of
classes is high to improve the performance of clustering. We observe that as we
add more attributes, the clustering performance increases overall. In the
second problem, we study the role of clustering in aggregating templates in a
1:N open set protocol using multi-shot video as a probe. We observe that by
increasing the number of clusters, the performance increases with respect to
the baseline and reaches a peak, after which increasing the number of clusters
causes the performance to degrade. Experiments are conducted using recently
introduced unconstrained IARPA Janus IJB-A, CS2, and CS3 face recognition
datasets.
Limits of End-to-End Learning
Authors: Tobias Glasmachers
Date: 2017-04-26 00:00:00
Tasks: Representation Learning

End-to-end learning refers to training a possibly complex learning system by
applying gradient-based learning to the system as a whole. End-to-end learning
system is specifically designed so that all modules are differentiable. In
effect, not only a central learning machine, but also all "peripheral" modules
like representation learning and memory formation are covered by a holistic
learning process. The power of end-to-end learning has been demonstrated on
many tasks, like playing a whole array of Atari video games with a single
architecture. While pushing for solutions to more challenging tasks, network
architectures keep growing more and more complex.
  In this paper we ask the question whether and to what extent end-to-end
learning is a future-proof technique in the sense of scaling to complex and
diverse data processing architectures. We point out potential inefficiencies,
and we argue in particular that end-to-end learning does not make optimal use
of the modular design of present neural networks. Our surprisingly simple
experiments demonstrate these inefficiencies, up to the complete breakdown of
learning.
Diversity driven Attention Model for Query-based Abstractive Summarization
Authors: Preksha Nema, Mitesh Khapra, Anirban Laha, Balaraman Ravindran
Date: 2017-04-26 00:00:00
Proceeding: ACL 2017 7
Tasks: Abstractive Text Summarization, Diversity, Extractive Summarization, Machine Translation, Query-Based Extractive Summarization, Translation

Abstractive summarization aims to generate a shorter version of the document
covering all the salient points in a compact and coherent fashion. On the other
hand, query-based summarization highlights those points that are relevant in
the context of a given query. The encode-attend-decode paradigm has achieved
notable success in machine translation, extractive summarization, dialog
systems, etc. But it suffers from the drawback of generation of repeated
phrases. In this work we propose a model for the query-based summarization task
based on the encode-attend-decode paradigm with two key additions (i) a query
attention model (in addition to document attention model) which learns to focus
on different portions of the query at different time steps (instead of using a
static representation for the query) and (ii) a new diversity based attention
model which aims to alleviate the problem of repeating phrases in the summary.
In order to enable the testing of this model we introduce a new query-based
summarization dataset building on debatepedia. Our experiments show that with
these two additions the proposed model clearly outperforms vanilla
encode-attend-decode models with a gain of 28% (absolute) in ROUGE-L scores.
Deep Cross-Modal Audio-Visual Generation
Authors: Lele Chen, Sudhanshu Srivastava, Zhiyao Duan, Chenliang Xu
Date: 2017-04-26 00:00:00

Cross-modal audio-visual perception has been a long-lasting topic in
psychology and neurology, and various studies have discovered strong
correlations in human perception of auditory and visual stimuli. Despite works
in computational multimodal modeling, the problem of cross-modal audio-visual
generation has not been systematically studied in the literature. In this
paper, we make the first attempt to solve this cross-modal generation problem
leveraging the power of deep generative adversarial training. Specifically, we
use conditional generative adversarial networks to achieve cross-modal
audio-visual generation of musical performances. We explore different encoding
methods for audio and visual signals, and work on two scenarios:
instrument-oriented generation and pose-oriented generation. Being the first to
explore this new problem, we compose two new datasets with pairs of images and
sounds of musical performances of different instruments. Our experiments using
both classification and human evaluations demonstrate that our model has the
ability to generate one modality, i.e., audio/visual, from the other modality,
i.e., visual/audio, to a good extent. Our experiments on various design choices
along with the datasets will facilitate future research in this new problem
space.
Pruning variable selection ensembles
Authors: Chunxia Zhang, Yilei Wu, Mu Zhu
Date: 2017-04-26 00:00:00
Tasks: Diversity, Ensemble Learning, Variable Selection

In the context of variable selection, ensemble learning has gained increasing
interest due to its great potential to improve selection accuracy and to reduce
false discovery rate. A novel ordering-based selective ensemble learning
strategy is designed in this paper to obtain smaller but more accurate
ensembles. In particular, a greedy sorting strategy is proposed to rearrange
the order by which the members are included into the integration process.
Through stopping the fusion process early, a smaller subensemble with higher
selection accuracy can be obtained. More importantly, the sequential inclusion
criterion reveals the fundamental strength-diversity trade-off among ensemble
members. By taking stability selection (abbreviated as StabSel) as an example,
some experiments are conducted with both simulated and real-world data to
examine the performance of the novel algorithm. Experimental results
demonstrate that pruned StabSel generally achieves higher selection accuracy
and lower false discovery rates than StabSel and several other benchmark
methods.
Learning an Optimization Algorithm through Human Design Iterations
Authors: Thurston Sexton, Max Yi Ren
Date: 2016-08-24 00:00:00
Tasks: Bayesian Optimization

Solving optimal design problems through crowdsourcing faces a dilemma: On one
hand, human beings have been shown to be more effective than algorithms at
searching for good solutions of certain real-world problems with
high-dimensional or discrete solution spaces; on the other hand, the cost of
setting up crowdsourcing environments, the uncertainty in the crowd's
domain-specific competence, and the lack of commitment of the crowd, all
contribute to the lack of real-world application of design crowdsourcing. We
are thus motivated to investigate a solution-searching mechanism where an
optimization algorithm is tuned based on human demonstrations on solution
searching, so that the search can be continued after human participants abandon
the problem. To do so, we model the iterative search process as a Bayesian
Optimization (BO) algorithm, and propose an inverse BO (IBO) algorithm to find
the maximum likelihood estimators of the BO parameters based on human
solutions. We show through a vehicle design and control problem that the search
performance of BO can be improved by recovering its parameters based on an
effective human search. Thus, IBO has the potential to improve the success rate
of design crowdsourcing activities, by requiring only good search strategies
instead of good solutions from the crowd.
Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network
Authors: Aditya Gilra, Wulfram Gerstner
Date: 2017-02-21 00:00:00
Tasks: Open-Ended Question Answering

Brains need to predict how the body reacts to motor commands. It is an open
question how networks of spiking neurons can learn to reproduce the non-linear
body dynamics caused by motor commands, using local, online and stable learning
rules. Here, we present a supervised learning scheme for the feedforward and
recurrent connections in a network of heterogeneous spiking neurons. The error
in the output is fed back through fixed random connections with a negative
gain, causing the network to follow the desired dynamics, while an online and
local rule changes the weights. The rule for Feedback-based Online Local
Learning Of Weights (FOLLOW) is local in the sense that weight changes depend
on the presynaptic activity and the error signal projected onto the
postsynaptic neuron. We provide examples of learning linear, non-linear and
chaotic dynamics, as well as the dynamics of a two-link arm. Using the Lyapunov
method, and under reasonable assumptions and approximations, we show that
FOLLOW learning is stable uniformly, with the error going to zero
asymptotically.