pwc-archive/papers-with-abstracts huggingface dataset:
An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning
Authors: Dhruv Malik, Malayandi Palaniappan, Jaime F. Fisac, Dylan Hadfield-Menell, Stuart Russell, Anca D. Dragan
Date: 2018-06-11 00:00:00
Proceeding: ICML 2018 7
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Our goal is for AI systems to correctly identify and act according to their
human user's objectives. Cooperative Inverse Reinforcement Learning (CIRL)
formalizes this value alignment problem as a two-player game between a human
and robot, in which only the human knows the parameters of the reward function:
the robot needs to learn them as the interaction unfolds. Previous work showed
that CIRL can be solved as a POMDP, but with an action space size exponential
in the size of the reward parameter space. In this work, we exploit a specific
property of CIRL---the human is a full information agent---to derive an
optimality-preserving modification to the standard Bellman update; this reduces
the complexity of the problem by an exponential factor and allows us to relax
CIRL's assumption of human rationality. We apply this update to a variety of
POMDP solvers and find that it enables us to scale CIRL to non-trivial
problems, with larger reward parameter spaces, and larger action spaces for
both robot and human. In solutions to these larger problems, the human exhibits
pedagogic (teaching) behavior, while the robot interprets it as such and
attains higher value for the human.
Adaptive MCMC via Combining Local Samplers
Authors: Kiarash Shaloudegi, András György
Date: 2018-06-11 00:00:00

Markov chain Monte Carlo (MCMC) methods are widely used in machine learning. One of the major problems with MCMC is the question of how to design chains that mix fast over the whole state space; in particular, how to select the parameters of an MCMC algorithm. Here we take a different approach and, similarly to parallel MCMC methods, instead of trying to find a single chain that samples from the whole distribution, we combine samples from several chains run in parallel, each exploring only parts of the state space (e.g., a few modes only). The chains are prioritized based on kernel Stein discrepancy, which provides a good measure of performance locally. The samples from the independent chains are combined using a novel technique for estimating the probability of different regions of the sample space. Experimental results demonstrate that the proposed algorithm may provide significant speedups in different sampling problems. Most importantly, when combined with the state-of-the-art NUTS algorithm as the base MCMC sampler, our method remained competitive with NUTS on sampling from unimodal distributions, while significantly outperforming state-of-the-art competitors on synthetic multimodal problems as well as on a challenging sensor localization task.
Compression of phase-only holograms with JPEG standard and deep learning
Authors: Shuming Jiao, Zhi Jin, Chenliang Chang, Changyuan Zhou, Wenbin Zou, Xia Li
Date: 2018-06-11 00:00:00

It is a critical issue to reduce the enormous amount of data in the
processing, storage and transmission of a hologram in digital format. In
photograph compression, the JPEG standard is commonly supported by almost every
system and device. It will be favorable if JPEG standard is applicable to
hologram compression, with advantages of universal compatibility. However, the
reconstructed image from a JPEG compressed hologram suffers from severe quality
degradation since some high frequency features in the hologram will be lost
during the compression process. In this work, we employ a deep convolutional
neural network to reduce the artifacts in a JPEG compressed hologram.
Simulation and experimental results reveal that our proposed "JPEG + deep
learning" hologram compression scheme can achieve satisfactory reconstruction
results for a computer-generated phase-only hologram after compression.
Greybox fuzzing as a contextual bandits problem
Authors: Ketan Patil, Aditya Kanade
Date: 2018-06-11 00:00:00
Tasks: Multi-Armed Bandits

Greybox fuzzing is one of the most useful and effective techniques for the
bug detection in large scale application programs. It uses minimal amount of
instrumentation. American Fuzzy Lop (AFL) is a popular coverage based
evolutionary greybox fuzzing tool. AFL performs extremely well in fuzz testing
large applications and finding critical vulnerabilities, but AFL involves a lot
of heuristics while deciding the favored test case(s), skipping test cases
during fuzzing, assigning fuzzing iterations to test case(s). In this work, we
aim at replacing the heuristics the AFL uses while assigning the fuzzing
iterations to a test case during the random fuzzing. We formalize this problem
as a `contextual bandit problem' and we propose an algorithm to solve this
problem. We have implemented our approach on top of the AFL. We modify the
AFL's heuristics with our learned model through the policy gradient method. Our
learning algorithm selects the multiplier of the number of fuzzing iterations
to be assigned to a test case during random fuzzing, given a fixed length
substring of the test case to be fuzzed. We fuzz the substring with this new
energy value and continuously updates the policy based upon the interesting
test cases it produces on fuzzing.
Chaining Mutual Information and Tightening Generalization Bounds
Authors: Amir R. Asadi, Emmanuel Abbe, Sergio Verdú
Date: 2018-06-11 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Generalization Bounds

Bounding the generalization error of learning algorithms has a long history, which yet falls short in explaining various generalization successes including those of deep learning. Two important difficulties are (i) exploiting the dependencies between the hypotheses, (ii) exploiting the dependence between the algorithm's input and output. Progress on the first point was made with the chaining method, originating from the work of Kolmogorov, and used in the VC-dimension bound. More recently, progress on the second point was made with the mutual information method by Russo and Zou '15. Yet, these two methods are currently disjoint. In this paper, we introduce a technique to combine the chaining and mutual information methods, to obtain a generalization bound that is both algorithm-dependent and that exploits the dependencies between the hypotheses. We provide an example in which our bound significantly outperforms both the chaining and the mutual information bounds. As a corollary, we tighten Dudley's inequality when the learning algorithm chooses its output from a small subset of hypotheses with high probability.
Eve: A Gradient Based Optimization Method with Locally and Globally Adaptive Learning Rates
Authors: Hiroaki Hayashi, Jayanth Koushik, Graham Neubig
Date: 2016-11-04 00:00:00
Tasks: General Classification, image-classification, Image Classification, Stochastic Optimization

Adaptive gradient methods for stochastic optimization adjust the learning
rate for each parameter locally. However, there is also a global learning rate
which must be tuned in order to get the best performance. In this paper, we
present a new algorithm that adapts the learning rate locally for each
parameter separately, and also globally for all parameters together.
Specifically, we modify Adam, a popular method for training deep learning
models, with a coefficient that captures properties of the objective function.
Empirically, we show that our method, which we call Eve, outperforms Adam and
other popular methods in training deep neural networks, like convolutional
neural networks for image classification, and recurrent neural networks for
language tasks.
Generative Adversarial Network Architectures For Image Synthesis Using Capsule Networks
Authors: Yash Upadhyay, Paul Schrater
Date: 2018-06-11 00:00:00
Tasks: Diversity, Generative Adversarial Network, Image Generation

In this paper, we propose Generative Adversarial Network (GAN) architectures
that use Capsule Networks for image-synthesis. Based on the principal of
positional-equivariance of features, Capsule Network's ability to encode
spatial relationships between the features of the image helps it become a more
powerful critic in comparison to Convolutional Neural Networks (CNNs) used in
current architectures for image synthesis. Our proposed GAN architectures learn
the data manifold much faster and therefore, synthesize visually accurate
images in significantly lesser number of training samples and training epochs
in comparison to GANs and its variants that use CNNs. Apart from analyzing the
quantitative results corresponding the images generated by different
architectures, we also explore the reasons for the lower coverage and diversity
explored by the GAN architectures that use CNN critics.
Cross-dataset Person Re-Identification Using Similarity Preserved Generative Adversarial Networks
Authors: Jianming Lv, Xintong Wang
Date: 2018-06-11 00:00:00
Tasks: Person Re-Identification

Person re-identification (Re-ID) aims to match the image frames which contain
the same person in the surveillance videos. Most of the Re-ID algorithms
conduct supervised training in some small labeled datasets, so directly
deploying these trained models to the real-world large camera networks may lead
to a poor performance due to underfitting. The significant difference between
the source training dataset and the target testing dataset makes it challenging
to incrementally optimize the model. To address this challenge, we propose a
novel solution by transforming the unlabeled images in the target domain to fit
the original classifier by using our proposed similarity preserved generative
adversarial networks model, SimPGAN. Specifically, SimPGAN adopts the
generative adversarial networks with the cycle consistency constraint to
transform the unlabeled images in the target domain to the style of the source
domain. Meanwhile, SimPGAN uses the similarity consistency loss, which is
measured by a siamese deep convolutional neural network, to preserve the
similarity of the transformed images of the same person. Comprehensive
experiments based on multiple real surveillance datasets are conducted, and the
results show that our algorithm is better than the state-of-the-art
cross-dataset unsupervised person Re-ID algorithms.
The Effect of Network Width on the Performance of Large-batch Training
Authors: Lingjiao Chen, Hongyi Wang, Jinman Zhao, Dimitris Papailiopoulos, Paraschos Koutris
Date: 2018-06-11 00:00:00
Proceeding: NeurIPS 2018 12

Distributed implementations of mini-batch stochastic gradient descent (SGD)
suffer from communication overheads, attributed to the high frequency of
gradient updates inherent in small-batch training. Training with large batches
can reduce these overheads; however, large batches can affect the convergence
properties and generalization performance of SGD. In this work, we take a first
step towards analyzing how the structure (width and depth) of a neural network
affects the performance of large-batch training. We present new theoretical
results which suggest that--for a fixed number of parameters--wider networks
are more amenable to fast large-batch training compared to deeper ones. We
provide extensive experiments on residual and fully-connected neural networks
which suggest that wider networks can be trained using larger batches without
incurring a convergence slow-down, unlike their deeper variants.
DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
Authors: Wei He, Kai Liu, Jing Liu, Yajuan Lyu, Shiqi Zhao, Xinyan Xiao, Yu-An Liu, Yizhong Wang, Hua Wu, Qiaoqiao She, Xuan Liu, Tian Wu, Haifeng Wang
Date: 2017-11-14 00:00:00
Proceeding: WS 2018 7
Tasks: Machine Reading Comprehension, Reading Comprehension

This paper introduces DuReader, a new large-scale, open-domain Chinese ma-
chine reading comprehension (MRC) dataset, designed to address real-world MRC.
DuReader has three advantages over previous MRC datasets: (1) data sources:
questions and documents are based on Baidu Search and Baidu Zhidao; answers are
manually generated. (2) question types: it provides rich annotations for more
question types, especially yes-no and opinion questions, that leaves more
opportunity for the research community. (3) scale: it contains 200K questions,
420K answers and 1M documents; it is the largest Chinese MRC dataset so far.
Experiments show that human performance is well above current state-of-the-art
baseline systems, leaving plenty of room for the community to make
improvements. To help the community make these improvements, both DuReader and
baseline systems have been posted online. We also organize a shared competition
to encourage the exploration of more models. Since the release of the task,
there are significant improvements over the baselines.
Assumed Density Filtering Q-learning
Authors: Heejin Jeong, Clark Zhang, George J. Pappas, Daniel D. Lee
Date: 2017-12-09 00:00:00
Tasks: Atari Games, Bayesian Inference, Q-Learning, Reinforcement Learning

While off-policy temporal difference (TD) methods have widely been used in reinforcement learning due to their efficiency and simple implementation, their Bayesian counterparts have not been utilized as frequently. One reason is that the non-linear max operation in the Bellman optimality equation makes it difficult to define conjugate distributions over the value functions. In this paper, we introduce a novel Bayesian approach to off-policy TD methods, called as ADFQ, which updates beliefs on state-action values, Q, through an online Bayesian inference method known as Assumed Density Filtering. We formulate an efficient closed-form solution for the value update by approximately estimating analytic parameters of the posterior of the Q-beliefs. Uncertainty measures in the beliefs not only are used in exploration but also provide a natural regularization for the value update considering all next available actions. ADFQ converges to Q-learning as the uncertainty measures of the Q-beliefs decrease and improves common drawbacks of other Bayesian RL algorithms such as computational complexity. We extend ADFQ with a neural network. Our empirical results demonstrate that ADFQ outperforms comparable algorithms on various Atari 2600 games, with drastic improvements in highly stochastic domains or domains with a large action space.
DOOBNet: Deep Object Occlusion Boundary Detection from an Image
Authors: Guoxia Wang, Xiaohui Liang, Frederick W. B. Li
Date: 2018-06-11 00:00:00
Tasks: Boundary Detection, Decoder, Object

Object occlusion boundary detection is a fundamental and crucial research
problem in computer vision. This is challenging to solve as encountering the
extreme boundary/non-boundary class imbalance during training an object
occlusion boundary detector. In this paper, we propose to address this class
imbalance by up-weighting the loss contribution of false negative and false
positive examples with our novel Attention Loss function. We also propose a
unified end-to-end multi-task deep object occlusion boundary detection network
(DOOBNet) by sharing convolutional features to simultaneously predict object
boundary and occlusion orientation. DOOBNet adopts an encoder-decoder structure
with skip connection in order to automatically learn multi-scale and
multi-level features. We significantly surpass the state-of-the-art on the PIOD
dataset (ODS F-score of .702) and the BSDS ownership dataset (ODS F-score of
.555), as well as improving the detecting speed to as 0.037s per image on the
PIOD dataset.
Smoothed analysis of the low-rank approach for smooth semidefinite programs
Authors: Thomas Pumir, Samy Jelassi, Nicolas Boumal
Date: 2018-06-11 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Retrieval

We consider semidefinite programs (SDPs) of size n with equality constraints.
In order to overcome scalability issues, Burer and Monteiro proposed a
factorized approach based on optimizing over a matrix Y of size $n$ by $k$ such
that $X = YY^*$ is the SDP variable. The advantages of such formulation are
twofold: the dimension of the optimization variable is reduced and positive
semidefiniteness is naturally enforced. However, the problem in Y is
non-convex. In prior work, it has been shown that, when the constraints on the
factorized variable regularly define a smooth manifold, provided k is large
enough, for almost all cost matrices, all second-order stationary points
(SOSPs) are optimal. Importantly, in practice, one can only compute points
which approximately satisfy necessary optimality conditions, leading to the
question: are such points also approximately optimal? To this end, and under
similar assumptions, we use smoothed analysis to show that approximate SOSPs
for a randomly perturbed objective function are approximate global optima, with
k scaling like the square root of the number of constraints (up to log
factors). Moreover, we bound the optimality gap at the approximate solution of
the perturbed problem with respect to the original problem. We particularize
our results to an SDP relaxation of phase retrieval.
Leveraging translations for speech transcription in low-resource settings
Authors: Antonis Anastasopoulos, David Chiang
Date: 2018-03-23 00:00:00

Recently proposed data collection frameworks for endangered language
documentation aim not only to collect speech in the language of interest, but
also to collect translations into a high-resource language that will render the
collected resource interpretable. We focus on this scenario and explore whether
we can improve transcription quality under these extremely low-resource
settings with the assistance of text translations. We present a neural
multi-source model and evaluate several variations of it on three low-resource
datasets. We find that our multi-source model with shared attention outperforms
the baselines, reducing transcription character error rate by up to 12.3%.
Part-of-Speech Tagging on an Endangered Language: a Parallel Griko-Italian Resource
Authors: Antonis Anastasopoulos, Marika Lekakou, Josep Quer, Eleni Zimianiti, Justin DeBenedetto, David Chiang
Date: 2018-06-11 00:00:00
Proceeding: COLING 2018 8
Tasks: Active Learning, Cross-Lingual Transfer, Part-Of-Speech Tagging, POS, POS Tagging, Sentence

Most work on part-of-speech (POS) tagging is focused on high resource
languages, or examines low-resource and active learning settings through
simulated studies. We evaluate POS tagging techniques on an actual endangered
language, Griko. We present a resource that contains 114 narratives in Griko,
along with sentence-level translations in Italian, and provides gold
annotations for the test set. Based on a previously collected small corpus, we
investigate several traditional methods, as well as methods that take advantage
of monolingual data or project cross-lingual POS tags. We show that the
combination of a semi-supervised method with cross-lingual transfer is more
appropriate for this extremely challenging setting, with the best tagger
achieving an accuracy of 72.9%. With an applied active learning scheme, which
we use to collect sentence-level annotations over the test set, we achieve
improvements of more than 21 percentage points.
Robust Object Tracking with Crow Search Optimized Multi-cue Particle Filter
Authors: Kapil Sharma, Gurjit Singh Walia, Ashish Kumar, Astitwa Saxena, Kuldeep Singh
Date: 2018-06-11 00:00:00
Tasks: Object Tracking, Outlier Detection, State Estimation

Particle Filter(PF) is used extensively for estimation of target Non-linear
and Non-gaussian state. However, its performance suffers due to inherent
problem of sample degeneracy and impoverishment. In order to address this, we
propose a novel resampling method based upon Crow Search Optimization to
overcome low performing particles detected as outlier. Proposed outlier
detection mechanism with transductive reliability achieve faster convergence of
proposed PF tracking framework. In addition, we present an adaptive fuzzy
fusion model to integrate multi-cue extracted for each evaluated particle.
Automatic boosting and suppression of particles using proposed fusion model not
only enhances performance of resampling method but also achieve optimal state
estimation. Performance of the proposed tracker is evaluated over 12 benchmark
video sequences and compared with state-of-the-art solutions. Qualitative and
quantitative results reveals that the proposed tracker not only outperforms
existing solutions but also efficiently handle various tracking challenges. On
average of outcome, we achieve CLE of 7.98 and F-measure of 0.734.
Improving Transferability of Adversarial Examples with Input Diversity
Authors: Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jian-Yu Wang, Zhou Ren, Alan Yuille
Date: 2018-03-19 00:00:00
Proceeding: CVPR 2019 6
Tasks: Adversarial Attack, Diversity, Image Classification

Though CNNs have achieved the state-of-the-art performance on various vision tasks, they are vulnerable to adversarial examples --- crafted by adding human-imperceptible perturbations to clean images. However, most of the existing adversarial attacks only achieve relatively low success rates under the challenging black-box setting, where the attackers have no knowledge of the model structure and parameters. To this end, we propose to improve the transferability of adversarial examples by creating diverse input patterns. Instead of only using the original images to generate adversarial examples, our method applies random transformations to the input images at each iteration. Extensive experiments on ImageNet show that the proposed attack method can generate adversarial examples that transfer much better to different networks than existing baselines. By evaluating our method against top defense solutions and official baselines from NIPS 2017 adversarial competition, the enhanced attack reaches an average success rate of 73.0%, which outperforms the top-1 attack submission in the NIPS competition by a large margin of 6.6%. We hope that our proposed attack strategy can serve as a strong benchmark baseline for evaluating the robustness of networks to adversaries and the effectiveness of different defense methods in the future. Code is available at https://github.com/cihangxie/DI-2-FGSM.
A GPU-based WFST Decoder with Exact Lattice Generation
Authors: Zhehuai Chen, Justin Luitjens, Hainan Xu, Yiming Wang, Daniel Povey, Sanjeev Khudanpur
Date: 2018-04-09 00:00:00
Tasks: Decoder, GPU, Scheduling

We describe initial work on an extension of the Kaldi toolkit that supports
weighted finite-state transducer (WFST) decoding on Graphics Processing Units
(GPUs). We implement token recombination as an atomic GPU operation in order to
fully parallelize the Viterbi beam search, and propose a dynamic load balancing
strategy for more efficient token passing scheduling among GPU threads. We also
redesign the exact lattice generation and lattice pruning algorithms for better
utilization of the GPUs. Experiments on the Switchboard corpus show that the
proposed method achieves identical 1-best results and lattice quality in
recognition and confidence measure tasks, while running 3 to 15 times faster
than the single process Kaldi decoder. The above results are reported on
different GPU architectures. Additionally we obtain a 46-fold speedup with
sequence parallelism and multi-process service (MPS) in GPU.