pwc-archive/papers-with-abstracts huggingface dataset:
Discovering Implicational Knowledge in Wikidata
Authors: Tom Hanika, Maximilian Marx, Gerd Stumme
Date: 2019-02-03 00:00:00
Tasks: Knowledge Graphs

Knowledge graphs have recently become the state-of-the-art tool for
representing the diverse and complex knowledge of the world. Examples include
the proprietary knowledge graphs of companies such as Google, Facebook, IBM, or
Microsoft, but also freely available ones such as YAGO, DBpedia, and Wikidata.
A distinguishing feature of Wikidata is that the knowledge is collaboratively
edited and curated. While this greatly enhances the scope of Wikidata, it also
makes it impossible for a single individual to grasp complex connections
between properties or understand the global impact of edits in the graph. We
apply Formal Concept Analysis to efficiently identify comprehensible
implications that are implicitly present in the data. Although the complex
structure of data modelling in Wikidata is not amenable to a direct approach,
we overcome this limitation by extracting contextual representations of parts
of Wikidata in a systematic fashion. We demonstrate the practical feasibility
of our approach through several experiments and show that the results may lead
to the discovery of interesting implicational knowledge. Besides providing a
method for obtaining large real-world data sets for FCA, we sketch potential
applications in offering semantic assistance for editing and curating Wikidata.
Inferring Concept Hierarchies from Text Corpora via Hyperbolic Embeddings
Authors: Matt Le, Stephen Roller, Laetitia Papaxanthos, Douwe Kiela, Maximilian Nickel
Date: 2019-02-03 00:00:00
Proceeding: ACL 2019 7

We consider the task of inferring is-a relationships from large text corpora.
For this purpose, we propose a new method combining hyperbolic embeddings and
Hearst patterns. This approach allows us to set appropriate constraints for
inferring concept hierarchies from distributional contexts while also being
able to predict missing is-a relationships and to correct wrong extractions.
Moreover -- and in contrast with other methods -- the hierarchical nature of
hyperbolic space allows us to learn highly efficient representations and to
improve the taxonomic consistency of the inferred hierarchies. Experimentally,
we show that our approach achieves state-of-the-art performance on several
commonly-used benchmarks.
Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptions
Authors: Yunwen Lei, Ting Hu, Guiying Li, Ke Tang
Date: 2019-02-03 00:00:00

Stochastic gradient descent (SGD) is a popular and efficient method with wide applications in training deep neural nets and other nonconvex models. While the behavior of SGD is well understood in the convex learning setting, the existing theoretical results for SGD applied to nonconvex objective functions are far from mature. For example, existing results require to impose a nontrivial assumption on the uniform boundedness of gradients for all iterates encountered in the learning process, which is hard to verify in practical implementations. In this paper, we establish a rigorous theoretical foundation for SGD in nonconvex learning by showing that this boundedness assumption can be removed without affecting convergence rates. In particular, we establish sufficient conditions for almost sure convergence as well as optimal convergence rates for SGD applied to both general nonconvex objective functions and gradient-dominated objective functions. A linear convergence is further derived in the case with zero variances.
A Relational Tucker Decomposition for Multi-Relational Link Prediction
Authors: Yanjie Wang, Samuel Broscheit, Rainer Gemulla
Date: 2019-02-03 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Link Prediction

We propose the Relational Tucker3 (RT) decomposition for multi-relational
link prediction in knowledge graphs. We show that many existing knowledge graph
embedding models are special cases of the RT decomposition with certain
predefined sparsity patterns in its components. In contrast to these prior
models, RT decouples the sizes of entity and relation embeddings, allows
parameter sharing across relations, and does not make use of a predefined
sparsity pattern. We use the RT decomposition as a tool to explore whether it
is possible and beneficial to automatically learn sparsity patterns, and
whether dense models can outperform sparse models (using the same number of
parameters). Our experiments indicate that---depending on the dataset--both
questions can be answered affirmatively.
Online Diversity Control in Symbolic Regression via a Fast Hash-based Tree Similarity Measure
Authors: Bogdan Burlacu, Michael Affenzeller, Gabriel Kronberger, Michael Kommenda
Date: 2019-02-03 00:00:00
Tasks: Diversity, regression, Symbolic Regression

Diversity represents an important aspect of genetic programming, being
directly correlated with search performance. When considered at the genotype
level, diversity often requires expensive tree distance measures which have a
negative impact on the algorithm's runtime performance. In this work we
introduce a fast, hash-based tree distance measure to massively speed-up the
calculation of population diversity during the algorithmic run. We combine this
measure with the standard GA and the NSGA-II genetic algorithms to steer the
search towards higher diversity. We validate the approach on a collection of
benchmark problems for symbolic regression where our method consistently
outperforms the standard GA as well as NSGA-II configurations with different
secondary objectives.
An Empirical Study on Regularization of Deep Neural Networks by Local Rademacher Complexity
Authors: Yingzhen Yang, Jiahui Yu, Xingjian Li, Jun Huan, Thomas S. Huang
Date: 2019-02-03 00:00:00
Tasks: Neural Architecture Search

Regularization of Deep Neural Networks (DNNs) for the sake of improving their generalization capability is important and challenging. The development in this line benefits theoretical foundation of DNNs and promotes their usability in different areas of artificial intelligence. In this paper, we investigate the role of Rademacher complexity in improving generalization of DNNs and propose a novel regularizer rooted in Local Rademacher Complexity (LRC). While Rademacher complexity is well known as a distribution-free complexity measure of function class that help boost generalization of statistical learning methods, extensive study shows that LRC, its counterpart focusing on a restricted function class, leads to sharper convergence rates and potential better generalization given finite training sample. Our LRC based regularizer is developed by estimating the complexity of the function class centered at the minimizer of the empirical loss of DNNs. Experiments on various types of network architecture demonstrate the effectiveness of LRC regularization in improving generalization. Moreover, our method features the state-of-the-art result on the CIFAR-$10$ dataset with network architecture found by neural architecture search.
Quantum Speedup in Adaptive Boosting of Binary Classification
Authors: Ximing Wang, Yue-Chi Ma, Min-Hsiu Hsieh, Man-Hong Yung
Date: 2019-02-03 00:00:00
Tasks: BIG-bench Machine Learning, Binary Classification, Classification, General Classification, Quantum Machine Learning

In classical machine learning, a set of weak classifiers can be adaptively
combined to form a strong classifier for improving the overall performance, a
technique called adaptive boosting (or AdaBoost). However, constructing the
strong classifier for a large data set is typically resource consuming. Here we
propose a quantum extension of AdaBoost, demonstrating a quantum algorithm that
can output the optimal strong classifier with a quadratic speedup in the number
of queries of the weak classifiers. Our results also include a generalization
of the standard AdaBoost to the cases where the output of each classifier may
be probabilistic even for the same input. We prove that the update rules and
the query complexity of the non-deterministic classifiers are the same as those
of deterministic classifiers, which may be of independent interest to the
classical machine-learning community. Furthermore, the AdaBoost algorithm can
also be applied to data encoded in the form of quantum states; we show how the
training set can be simplified by using the tools of t-design. Our approach
describes a model of quantum machine learning where quantum speedup is achieved
in finding the optimal classifier, which can then be applied for classical
machine-learning applications.
Neural Extractive Text Summarization with Syntactic Compression
Authors: Jiacheng Xu, Greg Durrett
Date: 2019-02-03 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Document Summarization, Extractive Text Summarization, Text Summarization

Recent neural network approaches to summarization are largely either selection-based extraction or generation-based abstraction. In this work, we present a neural model for single-document summarization based on joint extraction and syntactic compression. Our model chooses sentences from the document, identifies possible compressions based on constituency parses, and scores those compressions with a neural model to produce the final summary. For learning, we construct oracle extractive-compressive summaries, then learn both of our components jointly with this supervision. Experimental results on the CNN/Daily Mail and New York Times datasets show that our model achieves strong performance (comparable to state-of-the-art systems) as evaluated by ROUGE. Moreover, our approach outperforms an off-the-shelf compression module, and human and manual evaluation shows that our model's output generally remains grammatical.
Night Time Haze and Glow Removal using Deep Dilated Convolutional Network
Authors: Shiba Kuanar, K. R. Rao, Dwarikanath Mahapatra, Monalisa Bilas
Date: 2019-02-03 00:00:00
Tasks: Single Image Haze Removal

In this paper, we address the single image haze removal problem in a
nighttime scene. The night haze removal is a severely ill-posed problem
especially due to the presence of various visible light sources with varying
colors and non-uniform illumination. These light sources are of different
shapes and introduce noticeable glow in night scenes. To address these effects
we introduce a deep learning based DeGlow-DeHaze iterative architecture which
accounts for varying color illumination and glows. First, our convolution
neural network (CNN) based DeGlow model is able to remove the glow effect
significantly and on top of it a separate DeHaze network is included to remove
the haze effect. For our recurrent network training, the hazy images and the
corresponding transmission maps are synthesized from the NYU depth datasets and
consequently restored a high-quality haze-free image. The experimental results
demonstrate that our hybrid CNN model outperforms other state-of-the-art
methods in terms of computation speed and image quality. We also show the
effectiveness of our model on a number of real images and compare our results
with the existing night haze heuristic models.
Value-aware Recommendation based on Reinforced Profit Maximization in E-commerce Systems
Authors: Changhua Pei, Xinru Yang, Qing Cui, Xiao Lin, Fei Sun, Peng Jiang, Wenwu Ou, Yongfeng Zhang
Date: 2019-02-03 00:00:00
Tasks: Recommendation Systems, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Existing recommendation algorithms mostly focus on optimizing traditional
recommendation measures, such as the accuracy of rating prediction in terms of
RMSE or the quality of top-$k$ recommendation lists in terms of precision,
recall, MAP, etc. However, an important expectation for commercial
recommendation systems is to improve the final revenue/profit of the system.
Traditional recommendation targets such as rating prediction and top-$k$
recommendation are not directly related to this goal. In this work, we blend
the fundamental concepts in online advertising and micro-economics into
personalized recommendation for profit maximization. Specifically, we propose
value-aware recommendation based on reinforcement learning, which directly
optimizes the economic value of candidate items to generate the recommendation
list. In particular, we generalize the basic concept of click conversion rate
(CVR) in computational advertising into the conversation rate of an arbitrary
user action (XVR) in E-commerce, where the user actions can be clicking, adding
to cart, adding to wishlist, etc. In this way, each type of user action is
mapped to its monetized economic value. Economic values of different user
actions are further integrated as the reward of a ranking list, and
reinforcement learning is used to optimize the recommendation list for the
maximum total value. Experimental results in both offline benchmarks and online
commercial systems verified the improved performance of our framework, in terms
of both traditional top-$k$ ranking tasks and the economic profits of the
system.
A Meta-MDP Approach to Exploration for Lifelong Reinforcement Learning
Authors: Francisco M. Garcia, Philip S. Thomas
Date: 2019-02-03 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper we consider the problem of how a reinforcement learning agent
that is tasked with solving a sequence of reinforcement learning problems (a
sequence of Markov decision processes) can use knowledge acquired early in its
lifetime to improve its ability to solve new problems. We argue that previous
experience with similar problems can provide an agent with information about
how it should explore when facing a new but related problem. We show that the
search for an optimal exploration strategy can be formulated as a reinforcement
learning problem itself and demonstrate that such strategy can leverage
patterns found in the structure of related problems. We conclude with
experiments that show the benefits of optimizing an exploration strategy using
our proposed approach.
Real-Time Freespace Segmentation on Autonomous Robots for Detection of Obstacles and Drop-Offs
Authors: Anish Singhani
Date: 2019-02-03 00:00:00
Tasks: GPU, Segmentation, Semantic Segmentation

Mobile robots navigating in indoor and outdoor environments must be able to
identify and avoid unsafe terrain. Although a significant amount of work has
been done on the detection of standing obstacles (solid obstructions), not much
work has been done on the detection of negative obstacles (e.g. dropoffs,
ledges, downward stairs). We propose a method of terrain safety segmentation
using deep convolutional networks. Our custom semantic segmentation
architecture uses a single camera as input and creates a freespace map
distinguishing safe terrain and obstacles. We then show how this freespace map
can be used for real-time navigation on an indoor robot. The results show that
our system generalizes well, is suitable for real-time operation, and runs at
around 55 fps on a small indoor robot powered by a low-power embedded GPU.
Automatic trajectory measurement of large numbers of crowded objects
Authors: Hui Li, Ye Liu, Yan Qiu Chen
Date: 2019-02-03 00:00:00

Complex motion patterns of natural systems, such as fish schools, bird
flocks, and cell groups, have attracted great attention from scientists for
years. Trajectory measurement of individuals is vital for quantitative and
high-throughput study of their collective behaviors. However, such data are
rare mainly due to the challenges of detection and tracking of large numbers of
objects with similar visual features and frequent occlusions. We present an
automatic and effective framework to measure trajectories of large numbers of
crowded oval-shaped objects, such as fish and cells. We first use a novel dual
ellipse locator to detect the coarse position of each individual and then
propose a variance minimization active contour method to obtain the optimal
segmentation results. For tracking, cost matrix of assignment between
consecutive frames is trainable via a random forest classifier with many
spatial, texture, and shape features. The optimal trajectories are found for
the whole image sequence by solving two linear assignment problems. We evaluate
the proposed method on many challenging data sets.
Quantitative Weak Convergence for Discrete Stochastic Processes
Authors: Xiang Cheng, Peter L. Bartlett, Michael. I. Jordan
Date: 2019-02-03 00:00:00

In this paper, we quantitative convergence in $W_2$ for a family of Langevin-like stochastic processes that includes stochastic gradient descent and related gradient-based algorithms. Under certain regularity assumptions, we show that the iterates of these stochastic processes converge to an invariant distribution at a rate of $\tilde{O}\lrp{1/\sqrt{k}}$ where $k$ is the number of steps; this rate is provably tight up to log factors. Our result reduces to a quantitative form of the classical Central Limit Theorem in the special case when the potential is quadratic.
Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigence
Authors: Dahyun Kim, Jihwan Bae, Yeonsik Jo, Jonghyun Choi
Date: 2019-02-03 00:00:00
Tasks: Incremental Learning, Transfer Learning

Incremental learning suffers from two challenging problems; forgetting of old
knowledge and intransigence on learning new knowledge. Prediction by the model
incrementally learned with a subset of the dataset are thus uncertain and the
uncertainty accumulates through the tasks by knowledge transfer. To prevent
overfitting to the uncertain knowledge, we propose to penalize confident
fitting to the uncertain knowledge by the Maximum Entropy Regularizer (MER).
Additionally, to reduce class imbalance and induce a self-paced curriculum on
new classes, we exclude a few samples from the new classes in every mini-batch,
which we call DropOut Sampling (DOS). We further rethink evaluation metrics for
forgetting and intransigence in incremental learning by tracking each sample's
confusion at the transition of a task since the existing metrics that compute
the difference in accuracy are often misleading. We show that the proposed
method, named 'MEDIC', outperforms the state-of-the-art incremental learning
algorithms in accuracy, forgetting, and intransigence measured by both the
existing and the proposed metrics by a large margin in extensive empirical
validations on CIFAR100 and a popular subset of ImageNet dataset
(TinyImageNet).
Review Conversational Reading Comprehension
Authors: Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
Date: 2019-02-03 00:00:00
Tasks: Language Modeling, Language Modelling, Machine Reading Comprehension, Reading Comprehension

Inspired by conversational reading comprehension (CRC), this paper studies a novel task of leveraging reviews as a source to build an agent that can answer multi-turn questions from potential consumers of online businesses. We first build a review CRC dataset and then propose a novel task-aware pre-tuning step running between language model (e.g., BERT) pre-training and domain-specific fine-tuning. The proposed pre-tuning requires no data annotation, but can greatly enhance the performance on our end task. Experimental results show that the proposed approach is highly effective and has competitive performance as the supervised approach. The dataset is available at \url{https://github.com/howardhsu/RCRC}
DeepPBM: Deep Probabilistic Background Model Estimation from Video Sequences
Authors: Amirreza Farnoosh, Behnaz Rezaei, Sarah Ostadabbas
Date: 2019-02-03 00:00:00

This paper presents a novel unsupervised probabilistic model estimation of
visual background in video sequences using a variational autoencoder framework.
Due to the redundant nature of the backgrounds in surveillance videos, visual
information of the background can be compressed into a low-dimensional subspace
in the encoder part of the variational autoencoder, while the highly variant
information of its moving foreground gets filtered throughout its
encoding-decoding process. Our deep probabilistic background model (DeepPBM)
estimation approach is enabled by the power of deep neural networks in learning
compressed representations of video frames and reconstructing them back to the
original domain. We evaluated the performance of our DeepPBM in background
subtraction on 9 surveillance videos from the background model challenge
(BMC2012) dataset, and compared that with a standard subspace learning
technique, robust principle component analysis (RPCA), which similarly
estimates a deterministic low dimensional representation of the background in
videos and is widely used for this application. Our method outperforms RPCA on
BMC2012 dataset with 23% in average in F-measure score, emphasizing that
background subtraction using the trained model can be done in more than 10
times faster.
Randomized Allocation with Nonparametric Estimation for Contextual Multi-Armed Bandits with Delayed Rewards
Authors: Sakshi Arya, Yuhong Yang
Date: 2019-02-03 00:00:00
Tasks: Multi-Armed Bandits

We study a multi-armed bandit problem with covariates in a setting where there is a possible delay in observing the rewards. Under some mild assumptions on the probability distributions for the delays and using an appropriate randomization to select the arms, the proposed strategy is shown to be strongly consistent.
Distributional property testing in a quantum world
Authors: András Gilyén, Tongyang Li
Date: 2019-02-02 00:00:00
Tasks: Learning Theory

A fundamental problem in statistics and learning theory is to test properties
of distributions. We show that quantum computers can solve such problems with
significant speed-ups. In particular, we give fast quantum algorithms for
testing closeness between unknown distributions, testing independence between
two distributions, and estimating the Shannon / von Neumann entropy of
distributions. The distributions can be either classical or quantum, however
our quantum algorithms require coherent quantum access to a process preparing
the samples. Our results build on the recent technique of quantum singular
value transformation, combined with more standard tricks such as
divide-and-conquer. The presented approach is a natural fit for distributional
property testing both in the classical and the quantum case, demonstrating the
first speed-ups for testing properties of density operators that can be
accessed coherently rather than only via sampling; for classical distributions
our algorithms significantly improve the precision dependence of some earlier
results.
Collaborative Sampling in Generative Adversarial Networks
Authors: Yuejiang Liu, Parth Kothari, Alexandre Alahi
Date: 2019-02-02 00:00:00
Tasks: Image Generation

The standard practice in Generative Adversarial Networks (GANs) discards the discriminator during sampling. However, this sampling method loses valuable information learned by the discriminator regarding the data distribution. In this work, we propose a collaborative sampling scheme between the generator and the discriminator for improved data generation. Guided by the discriminator, our approach refines the generated samples through gradient-based updates at a particular layer of the generator, shifting the generator distribution closer to the real data distribution. Additionally, we present a practical discriminator shaping method that can smoothen the loss landscape provided by the discriminator for effective sample refinement. Through extensive experiments on synthetic and image datasets, we demonstrate that our proposed method can improve generated samples both quantitatively and qualitatively, offering a new degree of freedom in GAN sampling.