pwc-archive/papers-with-abstracts huggingface dataset:
LIUBoost : Locality Informed Underboosting for Imbalanced Data Classification
Authors: Sajid Ahmed, Farshid Rayhan, Asif Mahbub, Md. Rafsan Jani, Swakkhar Shatabda, Dewan Md. Farid, Chowdhury Mofizur Rahman
Date: 2017-11-15 00:00:00
Tasks: Classification, General Classification

The problem of class imbalance along with class-overlapping has become a
major issue in the domain of supervised learning. Most supervised learning
algorithms assume equal cardinality of the classes under consideration while
optimizing the cost function and this assumption does not hold true for
imbalanced datasets which results in sub-optimal classification. Therefore,
various approaches, such as undersampling, oversampling, cost-sensitive
learning and ensemble based methods have been proposed for dealing with
imbalanced datasets. However, undersampling suffers from information loss,
oversampling suffers from increased runtime and potential overfitting while
cost-sensitive methods suffer due to inadequately defined cost assignment
schemes. In this paper, we propose a novel boosting based method called
LIUBoost. LIUBoost uses under sampling for balancing the datasets in every
boosting iteration like RUSBoost while incorporating a cost term for every
instance based on their hardness into the weight update formula minimizing the
information loss introduced by undersampling. LIUBoost has been extensively
evaluated on 18 imbalanced datasets and the results indicate significant
improvement over existing best performing method RUSBoost.
Can CNN Construct Highly Accurate Models Efficiently for High-Dimensional Problems in Complex Product Designs?
Authors: Yu Li, Hu Wang, Juanjuan Liu
Date: 2017-11-15 00:00:00

With the increase of the nonlinearity and dimension, it is difficult for the
present popular metamodeling techniques to construct reliable metamodels. To
address this problem, Convolutional Neural Network (CNN) is introduced to
construct a highly accurate metamodel efficiently. Considering the inherent
characteristics of the CNN, it is a potential modeling tool to handle highly
nonlinear and dimensional problems (hundreds-dimensional problems) with the
limited training samples. In order to evaluate the proposed CNN metamodel for
hundreds-dimensional and strong nonlinear problems, CNN is compared with other
metamodeling techniques. Furthermore, several high-dimensional analytical
functions are also employed to test the CNN metamodel. Testing and comparisons
confirm the efficiency and capability of the CNN metamodel for
hundreds-dimensional and strong nonlinear problems. Moreover, the proposed CNN
metamodel is also applied to IsoGeometric Analysis (IGA)-based optimization
successfully.
A Deep Learning Approach for Expert Identification in Question Answering Communities
Authors: Chen Zheng, Shuangfei Zhai, Zhongfei Zhang
Date: 2017-11-14 00:00:00
Tasks: Deep Learning, Question Answering, Sentence

In this paper, we describe an effective convolutional neural network
framework for identifying the expert in question answering community. This
approach uses the convolutional neural network and combines user feature
representations with question feature representations to compute scores that
the user who gets the highest score is the expert on this question. Unlike
prior work, this method does not measure expert based on measure answer content
quality to identify the expert but only require question sentence and user
embedding feature to identify the expert. Remarkably, Our model can be applied
to different languages and different domains. The proposed framework is trained
on two datasets, The first dataset is Stack Overflow and the second one is
Zhihu. The Top-1 accuracy results of our experiments show that our framework
outperforms the best baseline framework for expert identification.
An inner-loop free solution to inverse problems using deep neural networks
Authors: Qi Wei, Kai Fan, Lawrence Carin, Katherine A. Heller
Date: 2017-09-06 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Denoising

We propose a new method that uses deep learning techniques to accelerate the
popular alternating direction method of multipliers (ADMM) solution for inverse
problems. The ADMM updates consist of a proximity operator, a least squares
regression that includes a big matrix inversion, and an explicit solution for
updating the dual variables. Typically, inner loops are required to solve the
first two sub-minimization problems due to the intractability of the prior and
the matrix inversion. To avoid such drawbacks or limitations, we propose an
inner-loop free update rule with two pre-trained deep convolutional
architectures. More specifically, we learn a conditional denoising auto-encoder
which imposes an implicit data-dependent prior/regularization on ground-truth
in the first sub-minimization problem. This design follows an empirical
Bayesian strategy, leading to so-called amortized inference. For matrix
inversion in the second sub-problem, we learn a convolutional neural network to
approximate the matrix inversion, i.e., the inverse mapping is learned by
feeding the input through the learned forward network. Note that training this
neural network does not require ground-truth or measurements, i.e., it is
data-independent. Extensive experiments on both synthetic data and real
datasets demonstrate the efficiency and accuracy of the proposed method
compared with the conventional ADMM solution using inner loops for solving
inverse problems.
School bus routing by maximizing trip compatibility
Authors: Ali Shafahi, Zhongxiang Wang, Ali Haghani
Date: 2017-11-01 00:00:00
Tasks: Blocking, Scheduling

School bus planning is usually divided into routing and scheduling due to the
complexity of solving them concurrently. However, the separation between these
two steps may lead to worse solutions with higher overall costs than that from
solving them together. When finding the minimal number of trips in the routing
problem, neglecting the importance of trip compatibility may increase the
number of buses actually needed in the scheduling problem. This paper proposes
a new formulation for the multi-school homogeneous fleet routing problem that
maximizes trip compatibility while minimizing total travel time. This
incorporates the trip compatibility for the scheduling problem in the routing
problem. Since the problem is inherently just a routing problem, finding a good
solution is not cumbersome. To compare the performance of the model with
traditional routing problems, we generate eight mid-size data sets. Through
importing the generated trips of the routing problems into the bus scheduling
(blocking) problem, it is shown that the proposed model uses up to 13% fewer
buses than the common traditional routing models.
InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
Authors: Yunzhu Li, Jiaming Song, Stefano Ermon
Date: 2017-03-26 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Imitation Learning

The goal of imitation learning is to mimic expert behavior without access to
an explicit reward signal. Expert demonstrations provided by humans, however,
often show significant variability due to latent factors that are typically not
explicitly modeled. In this paper, we propose a new algorithm that can infer
the latent structure of expert demonstrations in an unsupervised way. Our
method, built on top of Generative Adversarial Imitation Learning, can not only
imitate complex behaviors, but also learn interpretable and meaningful
representations of complex behavioral data, including visual demonstrations. In
the driving domain, we show that a model learned from human demonstrations is
able to both accurately reproduce a variety of behaviors and accurately
anticipate human actions using raw visual inputs. Compared with various
baselines, our method can better capture the latent structure underlying expert
demonstrations, often recovering semantically meaningful factors of variation
in the data.
On Optimal Generalizability in Parametric Learning
Authors: Ahmad Beirami, Meisam Razaviyayn, Shahin Shahrampour, Vahid Tarokh
Date: 2017-11-14 00:00:00
Proceeding: NeurIPS 2017 12

We consider the parametric learning problem, where the objective of the
learner is determined by a parametric loss function. Employing empirical risk
minimization with possibly regularization, the inferred parameter vector will
be biased toward the training samples. Such bias is measured by the cross
validation procedure in practice where the data set is partitioned into a
training set used for training and a validation set, which is not used in
training and is left to measure the out-of-sample performance. A classical
cross validation strategy is the leave-one-out cross validation (LOOCV) where
one sample is left out for validation and training is done on the rest of the
samples that are presented to the learner, and this process is repeated on all
of the samples. LOOCV is rarely used in practice due to the high computational
complexity. In this paper, we first develop a computationally efficient
approximate LOOCV (ALOOCV) and provide theoretical guarantees for its
performance. Then we use ALOOCV to provide an optimization algorithm for
finding the regularizer in the empirical risk minimization framework. In our
numerical experiments, we illustrate the accuracy and efficiency of ALOOCV as
well as our proposed framework for the optimization of the regularizer.
Orthogonal Recurrent Neural Networks with Scaled Cayley Transform
Authors: Kyle Helfrich, Devin Willmott, Qiang Ye
Date: 2017-07-29 00:00:00
Proceeding: ICML 2018 7

Recurrent Neural Networks (RNNs) are designed to handle sequential data but
suffer from vanishing or exploding gradients. Recent work on Unitary Recurrent
Neural Networks (uRNNs) have been used to address this issue and in some cases,
exceed the capabilities of Long Short-Term Memory networks (LSTMs). We propose
a simpler and novel update scheme to maintain orthogonal recurrent weight
matrices without using complex valued matrices. This is done by parametrizing
with a skew-symmetric matrix using the Cayley transform. Such a parametrization
is unable to represent matrices with negative one eigenvalues, but this
limitation is overcome by scaling the recurrent weight matrix by a diagonal
matrix consisting of ones and negative ones. The proposed training scheme
involves a straightforward gradient calculation and update step. In several
experiments, the proposed scaled Cayley orthogonal recurrent neural network
(scoRNN) achieves superior results with fewer trainable parameters than other
unitary RNNs.
Reinforcement Learning Algorithm Selection
Authors: Romain Laroche, Raphael Feraud
Date: 2017-01-30 00:00:00
Proceeding: ICLR 2018 1
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper formalises the problem of online algorithm selection in the
context of Reinforcement Learning. The setup is as follows: given an episodic
task and a finite number of off-policy RL algorithms, a meta-algorithm has to
decide which RL algorithm is in control during the next episode so as to
maximize the expected return. The article presents a novel meta-algorithm,
called Epochal Stochastic Bandit Algorithm Selection (ESBAS). Its principle is
to freeze the policy updates at each epoch, and to leave a rebooted stochastic
bandit in charge of the algorithm selection. Under some assumptions, a thorough
theoretical analysis demonstrates its near-optimality considering the
structural sampling budget limitations. ESBAS is first empirically evaluated on
a dialogue task where it is shown to outperform each individual algorithm in
most configurations. ESBAS is then adapted to a true online setting where
algorithms update their policies after each transition, which we call SSBAS.
SSBAS is evaluated on a fruit collection task where it is shown to adapt the
stepsize parameter more efficiently than the classical hyperbolic decay, and on
an Atari game, where it improves the performance by a wide margin.
Multi-Advisor Reinforcement Learning
Authors: Romain Laroche, Mehdi Fatemi, Joshua Romoff, Harm van Seijen
Date: 2017-04-03 00:00:00
Proceeding: ICLR 2018 1
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We consider tackling a single-agent RL problem by distributing it to $n$
learners. These learners, called advisors, endeavour to solve the problem from
a different focus. Their advice, taking the form of action values, is then
communicated to an aggregator, which is in control of the system. We show that
the local planning method for the advisors is critical and that none of the
ones found in the literature is flawless: the egocentric planning overestimates
values of states where the other advisors disagree, and the agnostic planning
is inefficient around danger zones. We introduce a novel approach called
empathic and discuss its theoretical aspects. We empirically examine and
validate our theoretical findings on a fruit collection task.
An Accelerated Communication-Efficient Primal-Dual Optimization Framework for Structured Machine Learning
Authors: Chenxin Ma, Martin Jaggi, Frank E. Curtis, Nathan Srebro, Martin Takáč
Date: 2017-11-14 00:00:00
Tasks: BIG-bench Machine Learning, Distributed Optimization

Distributed optimization algorithms are essential for training machine
learning models on very large-scale datasets. However, they often suffer from
communication bottlenecks. Confronting this issue, a communication-efficient
primal-dual coordinate ascent framework (CoCoA) and its improved variant CoCoA+
have been proposed, achieving a convergence rate of $\mathcal{O}(1/t)$ for
solving empirical risk minimization problems with Lipschitz continuous losses.
In this paper, an accelerated variant of CoCoA+ is proposed and shown to
possess a convergence rate of $\mathcal{O}(1/t^2)$ in terms of reducing
suboptimality. The analysis of this rate is also notable in that the
convergence rate bounds involve constants that, except in extreme cases, are
significantly reduced compared to those previously provided for CoCoA+. The
results of numerical experiments are provided to show that acceleration can
lead to significant performance gains.
A causal framework for explaining the predictions of black-box sequence-to-sequence models
Authors: David Alvarez-Melis, Tommi S. Jaakkola
Date: 2017-07-06 00:00:00
Proceeding: EMNLP 2017 9

We interpret the predictions of any black-box structured input-structured
output model around a specific input-output pair. Our method returns an
"explanation" consisting of groups of input-output tokens that are causally
related. These dependencies are inferred by querying the black-box model with
perturbed inputs, generating a graph over tokens from the responses, and
solving a partitioning problem to select the most relevant components. We focus
the general approach on sequence-to-sequence problems, adopting a variational
autoencoder to yield meaningful input perturbations. We test our method across
several NLP sequence generation tasks.
Strategic Coalitions with Perfect Recall
Authors: Pavel Naumov, Jia Tao
Date: 2017-07-13 00:00:00

The paper proposes a bimodal logic that describes an interplay between
distributed knowledge modality and coalition know-how modality. Unlike other
similar systems, the one proposed here assumes perfect recall by all agents.
Perfect recall is captured in the system by a single axiom. The main technical
results are the soundness and the completeness theorems for the proposed
logical system.
Modeling Semantic Relatedness using Global Relation Vectors
Authors: Shoaib Jameel, Zied Bouraoui, Steven Schockaert
Date: 2017-11-14 00:00:00
Tasks: Relation

Word embedding models such as GloVe rely on co-occurrence statistics from a
large corpus to learn vector representations of word meaning. These vectors
have proven to capture surprisingly fine-grained semantic and syntactic
information. While we may similarly expect that co-occurrence statistics can be
used to capture rich information about the relationships between different
words, existing approaches for modeling such relationships have mostly relied
on manipulating pre-trained word vectors. In this paper, we introduce a novel
method which directly learns relation vectors from co-occurrence statistics. To
this end, we first introduce a variant of GloVe, in which there is an explicit
connection between word vectors and PMI weighted co-occurrence vectors. We then
show how relation vectors can be naturally embedded into the resulting vector
space.
Bayesian Optimization for Parameter Tuning of the XOR Neural Network
Authors: Lawrence Stewart, Mark Stalzer
Date: 2017-09-22 00:00:00
Tasks: Bayesian Optimization

When applying Machine Learning techniques to problems, one must select model
parameters to ensure that the system converges but also does not become stuck
at the objective function's local minimum. Tuning these parameters becomes a
non-trivial task for large models and it is not always apparent if the user has
found the optimal parameters. We aim to automate the process of tuning a Neural
Network, (where only a limited number of parameter search attempts are
available) by implementing Bayesian Optimization. In particular, by assigning
Gaussian Process Priors to the parameter space, we utilize Bayesian
Optimization to tune an Artificial Neural Network used to learn the XOR
function, with the result of achieving higher prediction accuracy.
C-WSL: Count-guided Weakly Supervised Localization
Authors: Mingfei Gao, Ang Li, Ruichi Yu, Vlad I. Morariu, Larry S. Davis
Date: 2017-11-14 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object

We introduce count-guided weakly supervised localization (C-WSL), an approach
that uses per-class object count as a new form of supervision to improve weakly
supervised localization (WSL). C-WSL uses a simple count-based region selection
algorithm to select high-quality regions, each of which covers a single object
instance during training, and improves existing WSL methods by training with
the selected regions. To demonstrate the effectiveness of C-WSL, we integrate
it into two WSL architectures and conduct extensive experiments on VOC2007 and
VOC2012. Experimental results show that C-WSL leads to large improvements in
WSL and that the proposed approach significantly outperforms the
state-of-the-art methods. The results of annotation experiments on VOC2007
suggest that a modest extra time is needed to obtain per-class object counts
compared to labeling only object categories in an image. Furthermore, we reduce
the annotation time by more than $2\times$ and $38\times$ compared to
center-click and bounding-box annotations.
Loss Functions for Multiset Prediction
Authors: Sean Welleck, Zixin Yao, Yu Gai, Jialin Mao, Zheng Zhang, Kyunghyun Cho
Date: 2017-11-14 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decision Making, Prediction, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

We study the problem of multiset prediction. The goal of multiset prediction
is to train a predictor that maps an input to a multiset consisting of multiple
items. Unlike existing problems in supervised learning, such as classification,
ranking and sequence generation, there is no known order among items in a
target multiset, and each item in the multiset may appear more than once,
making this problem extremely challenging. In this paper, we propose a novel
multiset loss function by viewing this problem from the perspective of
sequential decision making. The proposed multiset loss function is empirically
evaluated on two families of datasets, one synthetic and the other real, with
varying levels of difficulty, against various baseline loss functions including
reinforcement learning, sequence, and aggregated distribution matching loss
functions. The experiments reveal the effectiveness of the proposed loss
function over the others.
Stein Variational Gradient Descent as Gradient Flow
Authors: Qiang Liu
Date: 2017-04-25 00:00:00
Proceeding: NeurIPS 2017 12

Stein variational gradient descent (SVGD) is a deterministic sampling
algorithm that iteratively transports a set of particles to approximate given
distributions, based on an efficient gradient-based update that guarantees to
optimally decrease the KL divergence within a function space. This paper
develops the first theoretical analysis on SVGD, discussing its weak
convergence properties and showing that its asymptotic behavior is captured by
a gradient flow of the KL divergence functional under a new metric structure
induced by Stein operator. We also provide a number of results on Stein
operator and Stein's identity using the notion of weak derivative, including a
new proof of the distinguishability of Stein discrepancy under weak conditions.
Using Redescription Mining to Relate Clinical and Biological Characteristics of Cognitively Impaired and Alzheimer's Disease Patients
Authors: Matej Mihelčić, Goran Šimić, Mirjana Babić Leko, Nada Lavrač, Sašo Džeroski, Tomislav Šmuc
Date: 2017-02-20 00:00:00
Tasks: Attribute

We used redescription mining to find interpretable rules revealing
associations between those determinants that provide insights about the
Alzheimer's disease (AD). We extended the CLUS-RM redescription mining
algorithm to a constraint-based redescription mining (CBRM) setting, which
enables several modes of targeted exploration of specific, user-constrained
associations. Redescription mining enabled finding specific constructs of
clinical and biological attributes that describe many groups of subjects of
different size, homogeneity and levels of cognitive impairment. We confirmed
some previously known findings. However, in some instances, as with the
attributes: testosterone, the imaging attribute Spatial Pattern of
Abnormalities for Recognition of Early AD, as well as the levels of leptin and
angiopoietin-2 in plasma, we corroborated previously debatable findings or
provided additional information about these variables and their association
with AD pathogenesis. Applying redescription mining on ADNI data resulted with
the discovery of one largely unknown attribute: the Pregnancy-Associated
Protein-A (PAPP-A), which we found highly associated with cognitive impairment
in AD. Statistically significant correlations (p <= 0.01) were found between
PAPP-A and various different clinical tests. The high importance of this
finding lies in the fact that PAPP-A is a metalloproteinase, known to cleave
insulin-like growth factor binding proteins. Since it also shares similar
substrates with A Disintegrin and the Metalloproteinase family of enzymes that
act as {\alpha}-secretase to physiologically cleave amyloid precursor protein
(APP) in the non-amyloidogenic pathway, it could be directly involved in the
metabolism of APP very early during the disease course. Therefore, further
studies should investigate the role of PAPP-A in the development of AD more
thoroughly.
A visual search engine for Bangladeshi laws
Authors: Manash Kumar Mandal, Pinku Deb Nath, Arpeeta Shams Mizan, Nazmus Saquib
Date: 2017-11-14 00:00:00
Tasks: Articles, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Browsing and finding relevant information for Bangladeshi laws is a challenge
faced by all law students and researchers in Bangladesh, and by citizens who
want to learn about any legal procedure. Some law archives in Bangladesh are
digitized, but lack proper tools to organize the data meaningfully. We present
a text visualization tool that utilizes machine learning techniques to make the
searching of laws quicker and easier. Using Doc2Vec to layout law article
nodes, link mining techniques to visualize relevant citation networks, and
named entity recognition to quickly find relevant sections in long law
articles, our tool provides a faster and better search experience to the users.
Qualitative feedback from law researchers, students, and government officials
show promise for visually intuitive search tools in the context of
governmental, legal, and constitutional data in developing countries, where
digitized data does not necessarily pave the way towards an easy access to
information.