pwc-archive/papers-with-abstracts huggingface dataset:
Binarized Knowledge Graph Embeddings
Authors: Koki Kishimoto, Katsuhiko Hayashi, Genki Akai, Masashi Shimbo, Kazunori Komatani
Date: 2019-02-08 00:00:00
Tasks: Knowledge Graph Completion, Knowledge Graph Embeddings, Knowledge Graphs, Quantization, Tensor Decomposition

Tensor factorization has become an increasingly popular approach to knowledge
graph completion(KGC), which is the task of automatically predicting missing
facts in a knowledge graph. However, even with a simple model like
CANDECOMP/PARAFAC(CP) tensor decomposition, KGC on existing knowledge graphs is
impractical in resource-limited environments, as a large amount of memory is
required to store parameters represented as 32-bit or 64-bit floating point
numbers. This limitation is expected to become more stringent as existing
knowledge graphs, which are already huge, keep steadily growing in scale. To
reduce the memory requirement, we present a method for binarizing the
parameters of the CP tensor decomposition by introducing a quantization
function to the optimization problem. This method replaces floating
point-valued parameters with binary ones after training, which drastically
reduces the model size at run time. We investigate the trade-off between the
quality and size of tensor factorization models for several KGC benchmark
datasets. In our experiments, the proposed method successfully reduced the
model size by more than an order of magnitude while maintaining the task
performance. Moreover, a fast score computation technique can be developed with
bitwise operations.
Correlated bandits or: How to minimize mean-squared error online
Authors: Vinay Praneeth Boda, Prashanth L. A
Date: 2019-02-08 00:00:00

While the objective in traditional multi-armed bandit problems is to find the arm with the highest mean, in many settings, finding an arm that best captures information about other arms is of interest. This objective, however, requires learning the underlying correlation structure and not just the means of the arms. Sensors placement for industrial surveillance and cellular network monitoring are a few applications, where the underlying correlation structure plays an important role. Motivated by such applications, we formulate the correlated bandit problem, where the objective is to find the arm with the lowest mean-squared error (MSE) in estimating all the arms. To this end, we derive first an MSE estimator, based on sample variances and covariances, and show that our estimator exponentially concentrates around the true MSE. Under a best-arm identification framework, we propose a successive rejects type algorithm and provide bounds on the probability of error in identifying the best arm. Using minmax theory, we also derive fundamental performance limits for the correlated bandit problem.
Differentiable Physics-informed Graph Networks
Authors: Sungyong Seo, Yan Liu
Date: 2019-02-08 00:00:00
Tasks: Inductive Learning, Weather Forecasting

While physics conveys knowledge of nature built from an interplay between
observations and theory, it has been considered less importantly in deep neural
networks. Especially, there are few works leveraging physics behaviors when the
knowledge is given less explicitly. In this work, we propose a novel
architecture called Differentiable Physics-informed Graph Networks (DPGN) to
incorporate implicit physics knowledge which is given from domain experts by
informing it in latent space. Using the concept of DPGN, we demonstrate that
climate prediction tasks are significantly improved. Besides the experiment
results, we validate the effectiveness of the proposed module and provide
further applications of DPGN, such as inductive learning and multistep
predictions.
Can Genetic Programming Do Manifold Learning Too?
Authors: Andrew Lensen, Bing Xue, Mengjie Zhang
Date: 2019-02-08 00:00:00
Tasks: Dimensionality Reduction

Exploratory data analysis is a fundamental aspect of knowledge discovery that
aims to find the main characteristics of a dataset. Dimensionality reduction,
such as manifold learning, is often used to reduce the number of features in a
dataset to a manageable level for human interpretation. Despite this, most
manifold learning techniques do not explain anything about the original
features nor the true characteristics of a dataset. In this paper, we propose a
genetic programming approach to manifold learning called GP-MaL which evolves
functional mappings from a high-dimensional space to a lower dimensional space
through the use of interpretable trees. We show that GP-MaL is competitive with
existing manifold learning algorithms, while producing models that can be
interpreted and re-used on unseen data. A number of promising future directions
of research are found in the process.
EILearn: Learning Incrementally Using Previous Knowledge Obtained From an Ensemble of Classifiers
Authors: Shivang Agarwal, C. Ravindranath Chowdary, Shripriya Maheshwari
Date: 2019-02-08 00:00:00
Tasks: Incremental Learning

We propose an algorithm for incremental learning of classifiers. The proposed
method enables an ensemble of classifiers to learn incrementally by
accommodating new training data. We use an effective mechanism to overcome the
stability-plasticity dilemma. In incremental learning, the general convention
is to use only the knowledge acquired in the previous phase but not the
previously seen data. We follow this convention by retaining the previously
acquired knowledge which is relevant and using it along with the current data.
The performance of each classifier is monitored to eliminate the poorly
performing classifiers in the subsequent phases. Experimental results show that
the proposed approach outperforms the existing incremental learning approaches.
Understanding the One-Pixel Attack: Propagation Maps and Locality Analysis
Authors: Danilo Vasconcellos Vargas, Jiawei Su
Date: 2019-02-08 00:00:00

Deep neural networks were shown to be vulnerable to single pixel
modifications. However, the reason behind such phenomena has never been
elucidated. Here, we propose Propagation Maps which show the influence of the
perturbation in each layer of the network. Propagation Maps reveal that even in
extremely deep networks such as Resnet, modification in one pixel easily
propagates until the last layer. In fact, this initial local perturbation is
also shown to spread becoming a global one and reaching absolute difference
values that are close to the maximum value of the original feature maps in a
given layer. Moreover, we do a locality analysis in which we demonstrate that
nearby pixels of the perturbed one in the one-pixel attack tend to share the
same vulnerability, revealing that the main vulnerability lies in neither
neurons nor pixels but receptive fields. Hopefully, the analysis conducted in
this work together with a new technique called propagation maps shall shed
light into the inner workings of other adversarial samples and be the basis of
new defense systems to come.
Generating the support with extreme value losses
Authors: Nicholas Guttenberg
Date: 2019-02-08 00:00:00
Tasks: Prediction

When optimizing against the mean loss over a distribution of predictions in
the context of a regression task, then even if there is a distribution of
targets the optimal prediction distribution is always a delta function at a
single value. Methods of constructing generative models need to overcome this
tendency. We consider a simple method of summarizing the prediction error, such
that the optimal strategy corresponds to outputting a distribution of
predictions with a support that matches the support of the distribution of
targets --- optimizing against the minimal value of the loss given a set of
samples from the prediction distribution, rather than the mean. We show that
models trained against this loss learn to capture the support of the target
distribution and, when combined with an auxiliary classifier-like prediction
task, can be projected via rejection sampling to reproduce the full
distribution of targets. The resulting method works well compared to other
generative modeling approaches particularly in low dimensional spaces with
highly non-trivial distributions, due to mode collapse solutions being globally
suboptimal with respect to the extreme value loss. However, the method is less
suited to high-dimensional spaces such as images due to the scaling of the
number of samples needed in order to accurately estimate the extreme value loss
when the dimension of the data manifold becomes large.
Expressive mechanisms for equitable rent division on a budget
Authors: Rodrigo A. Velez
Date: 2019-02-08 00:00:00

We study the incentive properties of envy-free mechanisms for the allocation of rooms and payments of rent among financially constrained roommates. Each agent reports her values for rooms, her housing earmark (soft budget), and an index that reflects the difficulty the agent experiences from having to pay over this amount. Then an envy-free allocation for these reports is recommended. The complete information non-cooperative outcomes of each of these mechanisms are exactly the envy-free allocations with respect to true preferences if and only if the admissible budget violation indices have a bound.
Mode Collapse and Regularity of Optimal Transportation Maps
Authors: Na lei, Yang Guo, Dongsheng An, Xin Qi, Zhongxuan Luo, Shing-Tung Yau, Xianfeng GU
Date: 2019-02-08 00:00:00

This work builds the connection between the regularity theory of optimal
transportation map, Monge-Amp\`{e}re equation and GANs, which gives a theoretic
understanding of the major drawbacks of GANs: convergence difficulty and mode
collapse.
  According to the regularity theory of Monge-Amp\`{e}re equation, if the
support of the target measure is disconnected or just non-convex, the optimal
transportation mapping is discontinuous. General DNNs can only approximate
continuous mappings. This intrinsic conflict leads to the convergence
difficulty and mode collapse in GANs.
  We test our hypothesis that the supports of real data distribution are in
general non-convex, therefore the discontinuity is unavoidable using an
Autoencoder combined with discrete optimal transportation map (AE-OT framework)
on the CelebA data set. The testing result is positive. Furthermore, we propose
to approximate the continuous Brenier potential directly based on discrete
Brenier theory to tackle mode collapse. Comparing with existing method, this
method is more accurate and effective.
Multi-task Learning for Target-dependent Sentiment Classification
Authors: Divam Gupta, Kushagra Singh, Soumen Chakrabarti, Tanmoy Chakraborty
Date: 2019-02-08 00:00:00
Tasks: Classification, General Classification, Multi-Task Learning, Sentiment Analysis, Sentiment Classification, Word Embeddings

Detecting and aggregating sentiments toward people, organizations, and events
expressed in unstructured social media have become critical text mining
operations. Early systems detected sentiments over whole passages, whereas more
recently, target-specific sentiments have been of greater interest. In this
paper, we present MTTDSC, a multi-task target-dependent sentiment
classification system that is informed by feature representation learnt for the
related auxiliary task of passage-level sentiment classification. The auxiliary
task uses a gated recurrent unit (GRU) and pools GRU states, followed by an
auxiliary fully-connected layer that outputs passage-level predictions. In the
main task, these GRUs contribute auxiliary per-token representations over and
above word embeddings. The main task has its own, separate GRUs. The auxiliary
and main GRUs send their states to a different fully connected layer, trained
for the main task. Extensive experiments using two auxiliary datasets and three
benchmark datasets (of which one is new, introduced by us) for the main task
demonstrate that MTTDSC outperforms state-of-the-art baselines. Using
word-level sensitivity analysis, we present anecdotal evidence that prior
systems can make incorrect target-specific predictions because they miss
sentiments expressed by words independent of target.
A Single-shot Object Detector with Feature Aggragation and Enhancement
Authors: Weiqiang Li, Guizhong Liu
Date: 2019-02-08 00:00:00

For many real applications, it is equally important to detect objects accurately and quickly. In this paper, we propose an accurate and efficient single shot object detector with feature aggregation and enhancement (FAENet). Our motivation is to enhance and exploit the shallow and deep feature maps of the whole network simultaneously. To achieve it we introduce a pair of novel feature aggregation modules and two feature enhancement blocks, and integrate them into the original structure of SSD. Extensive experiments on both the PASCAL VOC and MS COCO datasets demonstrate that the proposed method achieves much higher accuracy than SSD. In addition, our method performs better than the state-of-the-art one-stage detector RefineDet on small objects and can run at a faster speed.
Informing Computer Vision with Optical Illusions
Authors: Nasim Nematzadeh, David M. W. Powers, Trent Lewis
Date: 2019-02-08 00:00:00

Illusions are fascinating and immediately catch people's attention and
interest, but they are also valuable in terms of giving us insights into human
cognition and perception. A good theory of human perception should be able to
explain the illusion, and a correct theory will actually give quantifiable
results. We investigate here the efficiency of a computational filtering model
utilised for modelling the lateral inhibition of retinal ganglion cells and
their responses to a range of Geometric Illusions using isotropic Differences
of Gaussian filters. This study explores the way in which illusions have been
explained and shows how a simple standard model of vision based on classical
receptive fields can predict the existence of these illusions as well as the
degree of effect. A fundamental contribution of this work is to link bottom-up
processes to higher level perception and cognition consistent with Marr's
theory of vision and edge map representation.
A Comprehensive Overview of Biometric Fusion
Authors: Maneet Singh, Richa Singh, Arun Ross
Date: 2019-02-08 00:00:00

The performance of a biometric system that relies on a single biometric
modality (e.g., fingerprints only) is often stymied by various factors such as
poor data quality or limited scalability. Multibiometric systems utilize the
principle of fusion to combine information from multiple sources in order to
improve recognition accuracy whilst addressing some of the limitations of
single-biometric systems. The past two decades have witnessed the development
of a large number of biometric fusion schemes. This paper presents an overview
of biometric fusion with specific focus on three questions: what to fuse, when
to fuse, and how to fuse. A comprehensive review of techniques incorporating
ancillary information in the biometric recognition pipeline is also presented.
In this regard, the following topics are discussed: (i) incorporating data
quality in the biometric recognition pipeline; (ii) combining soft biometric
attributes with primary biometric identifiers; (iii) utilizing contextual
information to improve biometric recognition accuracy; and (iv) performing
continuous authentication using ancillary information. In addition, the use of
information fusion principles for presentation attack detection and
multibiometric cryptosystems is also discussed. Finally, some of the research
challenges in biometric fusion are enumerated. The purpose of this article is
to provide readers a comprehensive overview of the role of information fusion
in biometrics.
Certified Adversarial Robustness via Randomized Smoothing
Authors: Jeremy M Cohen, Elan Rosenfeld, J. Zico Kolter
Date: 2019-02-08 00:00:00
Tasks: Adversarial Defense, Adversarial Robustness, Robust classification

We show how to turn any classifier that classifies well under Gaussian noise into a new classifier that is certifiably robust to adversarial perturbations under the $\ell_2$ norm. This "randomized smoothing" technique has been proposed recently in the literature, but existing guarantees are loose. We prove a tight robustness guarantee in $\ell_2$ norm for smoothing with Gaussian noise. We use randomized smoothing to obtain an ImageNet classifier with e.g. a certified top-1 accuracy of 49% under adversarial perturbations with $\ell_2$ norm less than 0.5 (=127/255). No certified defense has been shown feasible on ImageNet except for smoothing. On smaller-scale datasets where competing approaches to certified $\ell_2$ robustness are viable, smoothing delivers higher certified accuracies. Our strong empirical results suggest that randomized smoothing is a promising direction for future research into adversarially robust classification. Code and models are available at http://github.com/locuslab/smoothing.
AdaScale: Towards Real-time Video Object Detection Using Adaptive Scaling
Authors: Ting-Wu Chin, Ruizhou Ding, Diana Marculescu
Date: 2019-02-08 00:00:00
Tasks: object-detection, Object Detection, Video Object Detection

In vision-enabled autonomous systems such as robots and autonomous cars,
video object detection plays a crucial role, and both its speed and accuracy
are important factors to provide reliable operation. The key insight we show in
this paper is that speed and accuracy are not necessarily a trade-off when it
comes to image scaling. Our results show that re-scaling the image to a lower
resolution will sometimes produce better accuracy. Based on this observation,
we propose a novel approach, dubbed AdaScale, which adaptively selects the
input image scale that improves both accuracy and speed for video object
detection. To this end, our results on ImageNet VID and mini
YouTube-BoundingBoxes datasets demonstrate 1.3 points and 2.7 points mAP
improvement with 1.6x and 1.8x speedup, respectively. Additionally, we improve
state-of-the-art video acceleration work by an extra 1.25x speedup with
slightly better mAP on ImageNet VID dataset.
Source Traces for Temporal Difference Learning
Authors: Silviu Pitis
Date: 2019-02-08 00:00:00

This paper motivates and develops source traces for temporal difference (TD)
learning in the tabular setting. Source traces are like eligibility traces, but
model potential histories rather than immediate ones. This allows TD errors to
be propagated to potential causal states and leads to faster generalization.
Source traces can be thought of as the model-based, backward view of successor
representations (SR), and share many of the same benefits. This view, however,
suggests several new ideas. First, a TD($\lambda$)-like source learning
algorithm is proposed and its convergence is proven. Then, a novel algorithm
for learning the source map (or SR matrix) is developed and shown to outperform
the previous algorithm. Finally, various approaches to using the source/SR
model are explored, and it is shown that source traces can be effectively
combined with other model-based methods like Dyna and experience replay.
Modeling Heterogeneity in Mode-Switching Behavior Under a Mobility-on-Demand Transit System: An Interpretable Machine Learning Approach
Authors: Xilei Zhao, Xiang Yan, Pascal Van Hentenryck
Date: 2019-02-08 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Interpretable Machine Learning

Recent years have witnessed an increased focus on interpretability and the
use of machine learning to inform policy analysis and decision making. This
paper applies machine learning to examine travel behavior and, in particular,
on modeling changes in travel modes when individuals are presented with a novel
(on-demand) mobility option. It addresses the following question: Can machine
learning be applied to model individual taste heterogeneity (preference
heterogeneity for travel modes and response heterogeneity to travel attributes)
in travel mode choice? This paper first develops a high-accuracy classifier to
predict mode-switching behavior under a hypothetical Mobility-on-Demand Transit
system (i.e., stated-preference data), which represents the case study
underlying this research. We show that this classifier naturally captures
individual heterogeneity available in the data. Moreover, the paper derives
insights on heterogeneous switching behaviors through the generation of
marginal effects and elasticities by current travel mode, partial dependence
plots, and individual conditional expectation plots. The paper also proposes
two new model-agnostic interpretation tools for machine learning, i.e.,
conditional partial dependence plots and conditional individual partial
dependence plots, specifically designed to examine response heterogeneity. The
results on the case study show that the machine-learning classifier, together
with model-agnostic interpretation tools, provides valuable insights on travel
mode switching behavior for different individuals and population segments. For
example, the existing drivers are more sensitive to additional pickups than
people using other travel modes, and current transit users are generally
willing to share rides but reluctant to take any additional transfers.
Rethinking the Discount Factor in Reinforcement Learning: A Decision Theoretic Approach
Authors: Silviu Pitis
Date: 2019-02-08 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

Reinforcement learning (RL) agents have traditionally been tasked with
maximizing the value function of a Markov decision process (MDP), either in
continuous settings, with fixed discount factor $\gamma < 1$, or in episodic
settings, with $\gamma = 1$. While this has proven effective for specific tasks
with well-defined objectives (e.g., games), it has never been established that
fixed discounting is suitable for general purpose use (e.g., as a model of
human preferences). This paper characterizes rationality in sequential decision
making using a set of seven axioms and arrives at a form of discounting that
generalizes traditional fixed discounting. In particular, our framework admits
a state-action dependent "discount" factor that is not constrained to be less
than 1, so long as there is eventual long run discounting. Although this
broadens the range of possible preference structures in continuous settings, we
show that there exists a unique "optimizing MDP" with fixed $\gamma < 1$ whose
optimal value function matches the true utility of the optimal policy, and we
quantify the difference between value and utility for suboptimal policies. Our
work can be seen as providing a normative justification for (a slight
generalization of) Martha White's RL task formalism (2017) and other recent
departures from the traditional RL, and is relevant to task specification in
RL, inverse RL and preference-based RL.
Lower Bounds for Learning Distributions under Communication Constraints via Fisher Information
Authors: Leighton Pate Barnes, Yanjun Han, Ayfer Ozgur
Date: 2019-02-07 00:00:00

We consider the problem of learning high-dimensional, nonparametric and structured (e.g. Gaussian) distributions in distributed networks, where each node in the network observes an independent sample from the underlying distribution and can use $k$ bits to communicate its sample to a central processor. We consider three different models for communication. Under the independent model, each node communicates its sample to a central processor by independently encoding it into $k$ bits. Under the more general sequential or blackboard communication models, nodes can share information interactively but each node is restricted to write at most $k$ bits on the final transcript. We characterize the impact of the communication constraint $k$ on the minimax risk of estimating the underlying distribution under $\ell^2$ loss. We develop minimax lower bounds that apply in a unified way to many common statistical models and reveal that the impact of the communication constraint can be qualitatively different depending on the tail behavior of the score function associated with each model. A key ingredient in our proofs is a geometric characterization of Fisher information from quantized samples.