pwc-archive/papers-with-abstracts huggingface dataset:
A Quantum-Inspired Ensemble Method and Quantum-Inspired Forest Regressors
Authors: Zeke Xie, Issei Sato
Date: 2017-11-22 00:00:00
Tasks: Diversity, regression

We propose a Quantum-Inspired Subspace(QIS) Ensemble Method for generating
feature ensembles based on feature selections. We assign each principal
component a Fraction Transition Probability as its probability weight based on
Principal Component Analysis and quantum interpretations. In order to generate
the feature subset for each base regressor, we select a feature subset from
principal components based on Fraction Transition Probabilities. The idea
originating from quantum mechanics can encourage ensemble diversity and the
accuracy simultaneously. We incorporate Quantum-Inspired Subspace Method into
Random Forest and propose Quantum-Inspired Forest. We theoretically prove that
the quantum interpretation corresponds to the first order approximation of
ensemble regression. We also evaluate the empirical performance of
Quantum-Inspired Forest and Random Forest in multiple hyperparameter settings.
Quantum-Inspired Forest proves the significant robustness of the default
hyperparameters on most data sets. The contribution of this work is two-fold, a
novel ensemble regression algorithm inspired by quantum mechanics and the
theoretical connection between quantum interpretations and machine learning
algorithms.
From virtual demonstration to real-world manipulation using LSTM and MDN
Authors: Rouhollah Rahmatizadeh, Pooya Abolghasemi, Aman Behal, Ladislau Bölöni
Date: 2016-03-12 00:00:00

Robots assisting the disabled or elderly must perform complex manipulation
tasks and must adapt to the home environment and preferences of their user.
Learning from demonstration is a promising choice, that would allow the
non-technical user to teach the robot different tasks. However, collecting
demonstrations in the home environment of a disabled user is time consuming,
disruptive to the comfort of the user, and presents safety challenges. It would
be desirable to perform the demonstrations in a virtual environment. In this
paper we describe a solution to the challenging problem of behavior transfer
from virtual demonstration to a physical robot. The virtual demonstrations are
used to train a deep neural network based controller, which is using a Long
Short Term Memory (LSTM) recurrent neural network to generate trajectories. The
training process uses a Mixture Density Network (MDN) to calculate an error
signal suitable for the multimodal nature of demonstrations. The controller
learned in the virtual environment is transferred to a physical robot (a
Rethink Robotics Baxter). An off-the-shelf vision component is used to
substitute for geometric knowledge available in the simulation and an inverse
kinematics module is used to allow the Baxter to enact the trajectory. Our
experimental studies validate the three contributions of the paper: (1) the
controller learned from virtual demonstrations can be used to successfully
perform the manipulation tasks on a physical robot, (2) the LSTM+MDN
architectural choice outperforms other choices, such as the use of feedforward
networks and mean-squared error based training signals and (3) allowing
imperfect demonstrations in the training set also allows the controller to
learn how to correct its manipulation mistakes.
Dual-Path Convolutional Image-Text Embeddings with Instance Loss
Authors: Zhedong Zheng, Liang Zheng, Michael Garrett, Yi Yang, Mingliang Xu, Yi-Dong Shen
Date: 2017-11-15 00:00:00
Tasks: Content-Based Image Retrieval, Cross-Modal Retrieval, NLP based Person Retrival, Person Retrieval, Retrieval, Text based Person Retrieval

Matching images and sentences demands a fine understanding of both modalities. In this paper, we propose a new system to discriminatively embed the image and text to a shared visual-textual space. In this field, most existing works apply the ranking loss to pull the positive image / text pairs close and push the negative pairs apart from each other. However, directly deploying the ranking loss is hard for network learning, since it starts from the two heterogeneous features to build inter-modal relationship. To address this problem, we propose the instance loss which explicitly considers the intra-modal data distribution. It is based on an unsupervised assumption that each image / text group can be viewed as a class. So the network can learn the fine granularity from every image/text group. The experiment shows that the instance loss offers better weight initialization for the ranking loss, so that more discriminative embeddings can be learned. Besides, existing works usually apply the off-the-shelf features, i.e., word2vec and fixed visual feature. So in a minor contribution, this paper constructs an end-to-end dual-path convolutional network to learn the image and text representations. End-to-end learning allows the system to directly learn from the data and fully utilize the supervision. On two generic retrieval datasets (Flickr30k and MSCOCO), experiments demonstrate that our method yields competitive accuracy compared to state-of-the-art methods. Moreover, in language based person retrieval, we improve the state of the art by a large margin. The code has been made publicly available.
Learning Discrete Distributions from Untrusted Batches
Authors: Mingda Qiao, Gregory Valiant
Date: 2017-11-22 00:00:00

We consider the problem of learning a discrete distribution in the presence
of an $\epsilon$ fraction of malicious data sources. Specifically, we consider
the setting where there is some underlying distribution, $p$, and each data
source provides a batch of $\ge k$ samples, with the guarantee that at least a
$(1-\epsilon)$ fraction of the sources draw their samples from a distribution
with total variation distance at most $\eta$ from $p$. We make no assumptions
on the data provided by the remaining $\epsilon$ fraction of sources--this data
can even be chosen as an adversarial function of the $(1-\epsilon)$ fraction of
"good" batches. We provide two algorithms: one with runtime exponential in the
support size, $n$, but polynomial in $k$, $1/\epsilon$ and $1/\eta$ that takes
$O((n+k)/\epsilon^2)$ batches and recovers $p$ to error
$O(\eta+\epsilon/\sqrt{k})$. This recovery accuracy is information
theoretically optimal, to constant factors, even given an infinite number of
data sources. Our second algorithm applies to the $\eta = 0$ setting and also
achieves an $O(\epsilon/\sqrt{k})$ recover guarantee, though it runs in
$\mathrm{poly}((nk)^k)$ time. This second algorithm, which approximates a
certain tensor via a rank-1 tensor minimizing $\ell_1$ distance, is surprising
in light of the hardness of many low-rank tensor approximation problems, and
may be of independent interest.
Interpreting CNN Knowledge via an Explanatory Graph
Authors: Quanshi Zhang, Ruiming Cao, Feng Shi, Ying Nian Wu, Song-Chun Zhu
Date: 2017-08-05 00:00:00
Tasks: Object

This paper learns a graphical model, namely an explanatory graph, which
reveals the knowledge hierarchy hidden inside a pre-trained CNN. Considering
that each filter in a conv-layer of a pre-trained CNN usually represents a
mixture of object parts, we propose a simple yet efficient method to
automatically disentangles different part patterns from each filter, and
construct an explanatory graph. In the explanatory graph, each node represents
a part pattern, and each edge encodes co-activation relationships and spatial
relationships between patterns. More importantly, we learn the explanatory
graph for a pre-trained CNN in an unsupervised manner, i.e., without a need of
annotating object parts. Experiments show that each graph node consistently
represents the same object part through different images. We transfer part
patterns in the explanatory graph to the task of part localization, and our
method significantly outperforms other approaches.
Examining CNN Representations with respect to Dataset Bias
Authors: Quanshi Zhang, Wenguan Wang, Song-Chun Zhu
Date: 2017-10-29 00:00:00
Tasks: Attribute

Given a pre-trained CNN without any testing samples, this paper proposes a
simple yet effective method to diagnose feature representations of the CNN. We
aim to discover representation flaws caused by potential dataset bias. More
specifically, when the CNN is trained to estimate image attributes, we mine
latent relationships between representations of different attributes inside the
CNN. Then, we compare the mined attribute relationships with ground-truth
attribute relationships to discover the CNN's blind spots and failure modes due
to dataset bias. In fact, representation flaws caused by dataset bias cannot be
examined by conventional evaluation strategies based on testing images, because
testing images may also have a similar bias. Experiments have demonstrated the
effectiveness of our method.
Deconvolutional Latent-Variable Model for Text Sequence Matching
Authors: Dinghan Shen, Yizhe Zhang, Ricardo Henao, Qinliang Su, Lawrence Carin
Date: 2017-09-21 00:00:00
Tasks: Decoder, model, Sentence, Text Matching

A latent-variable model is introduced for text matching, inferring sentence
representations by jointly optimizing generative and discriminative objectives.
To alleviate typical optimization challenges in latent-variable models for
text, we employ deconvolutional networks as the sequence decoder (generator),
providing learned latent codes with more semantic information and better
generalization. Our model, trained in an unsupervised manner, yields stronger
empirical predictive performance than a decoder based on Long Short-Term Memory
(LSTM), with less parameters and considerably faster training. Further, we
apply it to text sequence-matching problems. The proposed model significantly
outperforms several strong sentence-encoding baselines, especially in the
semi-supervised setting.
Visual Question Answering as a Meta Learning Task
Authors: Damien Teney, Anton Van Den Hengel
Date: 2017-11-22 00:00:00
Proceeding: ECCV 2018 9
Tasks: Meta-Learning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

The predominant approach to Visual Question Answering (VQA) demands that the
model represents within its weights all of the information required to answer
any question about any image. Learning this information from any real training
set seems unlikely, and representing it in a reasonable number of weights
doubly so. We propose instead to approach VQA as a meta learning task, thus
separating the question answering method from the information required. At test
time, the method is provided with a support set of example questions/answers,
over which it reasons to resolve the given question. The support set is not
fixed and can be extended without retraining, thereby expanding the
capabilities of the model. To exploit this dynamically provided information, we
adapt a state-of-the-art VQA model with two techniques from the recent meta
learning literature, namely prototypical networks and meta networks.
Experiments demonstrate the capability of the system to learn to produce
completely novel answers (i.e. never seen during training) from examples
provided at test time. In comparison to the existing state of the art, the
proposed method produces qualitatively distinct results with higher recall of
rare answers, and a better sample efficiency that allows training with little
initial data. More importantly, it represents an important step towards
vision-and-language methods that can learn and reason on-the-fly.
Asymmetric Action Abstractions for Multi-Unit Control in Adversarial Real-Time Games
Authors: Rubens O. Moraes, Levi H. S. Lelis
Date: 2017-11-22 00:00:00

Action abstractions restrict the number of legal actions available during
search in multi-unit real-time adversarial games, thus allowing algorithms to
focus their search on a set of promising actions. Optimal strategies derived
from un-abstracted spaces are guaranteed to be no worse than optimal strategies
derived from action-abstracted spaces. In practice, however, due to real-time
constraints and the state space size, one is only able to derive good
strategies in un-abstracted spaces in small-scale games. In this paper we
introduce search algorithms that use an action abstraction scheme we call
asymmetric abstraction. Asymmetric abstractions retain the un-abstracted
spaces' theoretical advantage over regularly abstracted spaces while still
allowing the search algorithms to derive effective strategies, even in
large-scale games. Empirical results on combat scenarios that arise in a
real-time strategy game show that our search algorithms are able to
substantially outperform state-of-the-art approaches.
Neurology-as-a-Service for the Developing World
Authors: Tejas Dharamsi, Payel Das, Tejaswini Pedapati, Gregory Bramble, Vinod Muthusamy, Horst Samulowitz, Kush R. Varshney, Yuvaraj Rajamanickam, John Thomas, Justin Dauwels
Date: 2017-11-16 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Feature Engineering

Electroencephalography (EEG) is an extensively-used and well-studied
technique in the field of medical diagnostics and treatment for brain
disorders, including epilepsy, migraines, and tumors. The analysis and
interpretation of EEGs require physicians to have specialized training, which
is not common even among most doctors in the developed world, let alone the
developing world where physician shortages plague society. This problem can be
addressed by teleEEG that uses remote EEG analysis by experts or by local
computer processing of EEGs. However, both of these options are prohibitively
expensive and the second option requires abundant computing resources and
infrastructure, which is another concern in developing countries where there
are resource constraints on capital and computing infrastructure. In this work,
we present a cloud-based deep neural network approach to provide decision
support for non-specialist physicians in EEG analysis and interpretation. Named
`neurology-as-a-service,' the approach requires almost no manual intervention
in feature engineering and in the selection of an optimal architecture and
hyperparameters of the neural network. In this study, we deploy a pipeline that
includes moving EEG data to the cloud and getting optimal models for various
classification tasks. Our initial prototype has been tested only in developed
world environments to-date, but our intention is to test it in developing world
environments in future work. We demonstrate the performance of our proposed
approach using the BCI2000 EEG MMI dataset, on which our service attains 63.4%
accuracy for the task of classifying real vs. imaginary activity performed by
the subject, which is significantly higher than what is obtained with a shallow
approach such as support vector machines.
The Emergence of Organizing Structure in Conceptual Representation
Authors: Brenden M. Lake, Neil D. Lawrence, Joshua B. Tenenbaum
Date: 2016-11-28 00:00:00
Tasks: Form, Inductive Bias

Both scientists and children make important structural discoveries, yet their
computational underpinnings are not well understood. Structure discovery has
previously been formalized as probabilistic inference about the right
structural form --- where form could be a tree, ring, chain, grid, etc. [Kemp &
Tenenbaum (2008). The discovery of structural form. PNAS, 105(3), 10687-10692].
While this approach can learn intuitive organizations, including a tree for
animals and a ring for the color circle, it assumes a strong inductive bias
that considers only these particular forms, and each form is explicitly
provided as initial knowledge. Here we introduce a new computational model of
how organizing structure can be discovered, utilizing a broad hypothesis space
with a preference for sparse connectivity. Given that the inductive bias is
more general, the model's initial knowledge shows little qualitative
resemblance to some of the discoveries it supports. As a consequence, the model
can also learn complex structures for domains that lack intuitive description,
as well as predict human property induction judgments without explicit
structural forms. By allowing form to emerge from sparsity, our approach
clarifies how both the richness and flexibility of human conceptual
organization can coexist.
Contrastive Principal Component Analysis
Authors: Abubakar Abid, Martin J. Zhang, Vivek K. Bagaria, James Zou
Date: 2017-09-20 00:00:00
Tasks: Denoising, feature selection, Subgroup Discovery

We present a new technique called contrastive principal component analysis
(cPCA) that is designed to discover low-dimensional structure that is unique to
a dataset, or enriched in one dataset relative to other data. The technique is
a generalization of standard PCA, for the setting where multiple datasets are
available -- e.g. a treatment and a control group, or a mixed versus a
homogeneous population -- and the goal is to explore patterns that are specific
to one of the datasets. We conduct a wide variety of experiments in which cPCA
identifies important dataset-specific patterns that are missed by PCA,
demonstrating that it is useful for many applications: subgroup discovery,
visualizing trends, feature selection, denoising, and data-dependent
standardization. We provide geometrical interpretations of cPCA and show that
it satisfies desirable theoretical guarantees. We also extend cPCA to nonlinear
settings in the form of kernel cPCA. We have released our code as a python
package and documentation is on Github.
Investigating Inner Properties of Multimodal Representation and Semantic Compositionality with Brain-based Componential Semantics
Authors: Shaonan Wang, Jiajun Zhang, Nan Lin, Cheng-qing Zong
Date: 2017-11-15 00:00:00
Tasks: Learning Semantic Representations, Natural Language Understanding

Multimodal models have been proven to outperform text-based approaches on
learning semantic representations. However, it still remains unclear what
properties are encoded in multimodal representations, in what aspects do they
outperform the single-modality representations, and what happened in the
process of semantic compositionality in different input modalities. Considering
that multimodal models are originally motivated by human concept
representations, we assume that correlating multimodal representations with
brain-based semantics would interpret their inner properties to answer the
above questions. To that end, we propose simple interpretation methods based on
brain-based componential semantics. First we investigate the inner properties
of multimodal representations by correlating them with corresponding
brain-based property vectors. Then we map the distributed vector space to the
interpretable brain-based componential space to explore the inner properties of
semantic compositionality. Ultimately, the present paper sheds light on the
fundamental questions of natural language understanding, such as how to
represent the meaning of words and how to combine word meanings into larger
units.
Application of Natural Language Processing to Determine User Satisfaction in Public Services
Authors: Radoslaw Kowalski, Marc Esteve, Slava J. Mikhaylov
Date: 2017-11-21 00:00:00
Tasks: Decision Making, Topic Models

Research on customer satisfaction has increased substantially in recent
years. However, the relative importance and relationships between different
determinants of satisfaction remains uncertain. Moreover, quantitative studies
to date tend to test for significance of pre-determined factors thought to have
an influence with no scalable means to identify other causes of user
satisfaction. The gaps in knowledge make it difficult to use available
knowledge on user preference for public service improvement. Meanwhile, digital
technology development has enabled new methods to collect user feedback, for
example through online forums where users can comment freely on their
experience. New tools are needed to analyze large volumes of such feedback. Use
of topic models is proposed as a feasible solution to aggregate open-ended user
opinions that can be easily deployed in the public sector. Generated insights
can contribute to a more inclusive decision-making process in public service
provision. This novel methodological approach is applied to a case of service
reviews of publicly-funded primary care practices in England. Findings from the
analysis of 145,000 reviews covering almost 7,700 primary care centers indicate
that the quality of interactions with staff and bureaucratic exigencies are the
key issues driving user satisfaction across England.
Robust Stackelberg Equilibria in Extensive-Form Games and Extension to Limited Lookahead
Authors: Christian Kroer, Gabriele Farina, Tuomas Sandholm
Date: 2017-11-21 00:00:00
Tasks: Form

Stackelberg equilibria have become increasingly important as a solution
concept in computational game theory, largely inspired by practical problems
such as security settings. In practice, however, there is typically uncertainty
regarding the model about the opponent. This paper is, to our knowledge, the
first to investigate Stackelberg equilibria under uncertainty in extensive-form
games, one of the broadest classes of game. We introduce robust Stackelberg
equilibria, where the uncertainty is about the opponent's payoffs, as well as
ones where the opponent has limited lookahead and the uncertainty is about the
opponent's node evaluation function. We develop a new mixed-integer program for
the deterministic limited-lookahead setting. We then extend the program to the
robust setting for Stackelberg equilibrium under unlimited and under limited
lookahead by the opponent. We show that for the specific case of interval
uncertainty about the opponent's payoffs (or about the opponent's node
evaluations in the case of limited lookahead), robust Stackelberg equilibria
can be computed with a mixed-integer program that is of the same asymptotic
size as that for the deterministic setting.
Towards a Mathematical Understanding of the Difficulty in Learning with Feedforward Neural Networks
Authors: Hao Shen
Date: 2016-11-17 00:00:00
Proceeding: CVPR 2018 6

Training deep neural networks for solving machine learning problems is one
great challenge in the field, mainly due to its associated optimisation problem
being highly non-convex. Recent developments have suggested that many training
algorithms do not suffer from undesired local minima under certain scenario,
and consequently led to great efforts in pursuing mathematical explanations for
such observations. This work provides an alternative mathematical understanding
of the challenge from a smooth optimisation perspective. By assuming exact
learning of finite samples, sufficient conditions are identified via a critical
point analysis to ensure any local minimum to be globally minimal as well.
Furthermore, a state of the art algorithm, known as the Generalised
Gauss-Newton (GGN) algorithm, is rigorously revisited as an approximate
Newton's algorithm, which shares the property of being locally quadratically
convergent to a global minimum under the condition of exact learning.