pwc-archive/papers-with-abstracts huggingface dataset:
Similarity-Aware Spectral Sparsification by Edge Filtering
Authors: Zhuo Feng
Date: 2017-11-14 00:00:00
Proceeding: ACM 2017 11

In recent years, spectral graph sparsification techniques that can compute ultra-sparse graph proxies have been extensively studied for accelerating various numerical and graph-related applications. Prior nearly-linear-time spectral sparsification methods first extract low-stretch spanning tree from the original graph to form the backbone of the sparsifier, and then recover small portions of spectrally-critical off-tree edges to the spanning tree to significantly improve the approximation quality. However, it is not clear how many off-tree edges should be recovered for achieving a desired spectral similarity level within the sparsifier. Motivated by recent graph signal processing techniques, this paper proposes a similarity-aware spectral graph sparsification framework that leverages efficient spectral off-tree edge embedding and filtering schemes to construct spectral sparsifiers with guaranteed spectral similarity (relative condition number) level. An iterative graph densification scheme is introduced to facilitate efficient and effective filtering of off-tree edges for highly ill-conditioned problems. The proposed method has been validated using various kinds of graphs obtained from public domain sparse matrix collections relevant to VLSI CAD, finite element analysis, as well as social and data networks frequently studied in many machine learning and data mining applications.
Attention Regularized Sequence-to-Sequence Learning for E2E NLG Challenge
Authors: Biao Zhang, Jing Yang, Qian Lin, Jinsong Su
Date: 2018-03-01 00:00:00
Proceeding: E2E NLG Challenge System Descriptions 2018 3
Tasks: Data-to-Text Generation, Text Generation

This paper describes our system used for the end-to-end (E2E) natural language generation (NLG) challenge. The challenge collects a novel dataset for spoken dialogue system in the restaurant domain, which shows more lexical richness and syntactic variation and requires content selection (Novikova et al., 2017). To solve this challenge, we employ the CAEncoder-enhanced sequence-tosequence learning model (Zhang et al., 2017) and propose an attention regularizer to spread attention weights across input words as well as control the overfitting problem. Without any specific designation, our system yields very promising performance. Particularly, our system achieves a ROUGE-L score of 0.7083, the best result among all submitted primary systems.
Technical Report for E2E NLG Challenge
Authors: Heng Gong
Date: 2017-12-19 00:00:00
Proceeding: E2E NLG Challenge System Descriptions 2017 12
Tasks: Data-to-Text Generation, Text Generation

This paper describes the primary system submitted by the author to the E2E NLG Challenge on the E2E Dataset (Novikova et al. (2017)). Based on the baseline system called TGen (Dusek and Jurcicek (2016)), the primary system uses REINFORCE to utilize multiple reference for single Meaning Representation during training, while the baseline model treated them as individual training instances.
TNT-NLG, System 1: Using a statistical NLG to massively augment crowd-sourced data for neural generation
Authors: Shereen Oraby, Lena Reed, Shubhangi Tandon, Stephanie Lukin, Marilyn A. Walker
Date: 2018-04-26 00:00:00
Proceeding: E2E NLG Challenge System Descriptions 2018 4
Tasks: Data-to-Text Generation, Machine Translation, Sentence, Text Generation, Translation

Ever since the successful application of sequence to sequence learning for neural machine translation systems (Sutskever et al., 2014), interest has surged in its applicability towards language generation in other problem domains. In the area of natural language generation (NLG), there has been a great deal of interest in end-to-end (E2E) neural models that learn and generate natural language sentence realizations in one step. In this paper, we present TNT-NLG System 1, our first system submission to the E2E NLG Challenge, where we generate natural language (NL) realizations from meaning representations (MRs) in the restaurant domain by massively expanding the training dataset. We develop two models for this system, based on Dusek et al.’s (2016a) open source baseline model and context-aware neural language generator. Starting with the MR and NL pairs from the E2E generation challenge dataset, we explode the size of the training set using PERSONAGE (Mairesse and Walker, 2010), a statistical generator able to produce varied realizations from MRs, and use our expanded data as contextual input into our models. We present evaluation results using automated and human evaluation metrics, and describe directions for future work.
Self-Attentive Sequential Recommendation
Authors: Wang-Cheng Kang, Julian McAuley
Date: 2018-08-20 00:00:00
Tasks: Recommendation Systems, Sequential Recommendation

Sequential dynamics are a key feature of many modern recommender systems,
which seek to capture the `context' of users' activities on the basis of
actions they have performed recently. To capture such patterns, two approaches
have proliferated: Markov Chains (MCs) and Recurrent Neural Networks (RNNs).
Markov Chains assume that a user's next action can be predicted on the basis of
just their last (or last few) actions, while RNNs in principle allow for
longer-term semantics to be uncovered. Generally speaking, MC-based methods
perform best in extremely sparse datasets, where model parsimony is critical,
while RNNs perform better in denser datasets where higher model complexity is
affordable. The goal of our work is to balance these two goals, by proposing a
self-attention based sequential model (SASRec) that allows us to capture
long-term semantics (like an RNN), but, using an attention mechanism, makes its
predictions based on relatively few actions (like an MC). At each time step,
SASRec seeks to identify which items are `relevant' from a user's action
history, and use them to predict the next item. Extensive empirical studies
show that our method outperforms various state-of-the-art sequential models
(including MC/CNN/RNN-based approaches) on both sparse and dense datasets.
Moreover, the model is an order of magnitude more efficient than comparable
CNN/RNN-based models. Visualizations on attention weights also show how our
model adaptively handles datasets with various density, and uncovers meaningful
patterns in activity sequences.
Unrestricted Adversarial Examples via Semantic Manipulation
Authors: Anand Bhattad, Min Jin Chong, Kaizhao Liang, Bo Li, D. A. Forsyth
Date: 2019-04-12 00:00:00
Proceeding: ICLR 2020 1
Tasks: Colorization, Image Captioning, image-classification, Image Classification

Machine learning models, especially deep neural networks (DNNs), have been shown to be vulnerable against adversarial examples which are carefully crafted samples with a small magnitude of the perturbation. Such adversarial perturbations are usually restricted by bounding their $\mathcal{L}_p$ norm such that they are imperceptible, and thus many current defenses can exploit this property to reduce their adversarial impact. In this paper, we instead introduce "unrestricted" perturbations that manipulate semantically meaningful image-based visual descriptors - color and texture - in order to generate effective and photorealistic adversarial examples. We show that these semantically aware perturbations are effective against JPEG compression, feature squeezing and adversarially trained model. We also show that the proposed methods can effectively be applied to both image classification and image captioning tasks on complex datasets such as ImageNet and MSCOCO. In addition, we conduct comprehensive user studies to show that our generated semantic adversarial examples are photorealistic to humans despite large magnitude perturbations when compared to other attacks.
Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation
Authors: Yuyin Zhou, Zhe Li, Song Bai, Chong Wang, Xinlei Chen, Mei Han, Elliot Fishman, Alan Yuille
Date: 2019-04-12 00:00:00
Proceeding: ICCV 2019 10
Tasks: Medical Image Segmentation, Organ Segmentation, Partially Labeled Datasets, Segmentation

Accurate multi-organ abdominal CT segmentation is essential to many clinical applications such as computer-aided intervention. As data annotation requires massive human labor from experienced radiologists, it is common that training data are partially labeled, e.g., pancreas datasets only have the pancreas labeled while leaving the rest marked as background. However, these background labels can be misleading in multi-organ segmentation since the "background" usually contains some other organs of interest. To address the background ambiguity in these partially-labeled datasets, we propose Prior-aware Neural Network (PaNN) via explicitly incorporating anatomical priors on abdominal organ sizes, guiding the training process with domain-specific knowledge. More specifically, PaNN assumes that the average organ size distributions in the abdomen should approximate their empirical distributions, a prior statistics obtained from the fully-labeled dataset. As our training objective is difficult to be directly optimized using stochastic gradient descent [20], we propose to reformulate it in a min-max form and optimize it via the stochastic primal-dual gradient algorithm. PaNN achieves state-of-the-art performance on the MICCAI2015 challenge "Multi-Atlas Labeling Beyond the Cranial Vault", a competition on organ segmentation in the abdomen. We report an average Dice score of 84.97%, surpassing the prior art by a large margin of 3.27%.
Incremental multi-domain learning with network latent tensor factorization
Authors: Adrian Bulat, Jean Kossaifi, Georgios Tzimiropoulos, Maja Pantic
Date: 2019-04-12 00:00:00
Tasks: General Classification, Image Classification, Incremental Learning, Tensor Networks

The prominence of deep learning, large amount of annotated data and increasingly powerful hardware made it possible to reach remarkable performance for supervised classification tasks, in many cases saturating the training sets. However the resulting models are specialized to a single very specific task and domain. Adapting the learned classification to new domains is a hard problem due to at least three reasons: (1) the new domains and the tasks might be drastically different; (2) there might be very limited amount of annotated data on the new domain and (3) full training of a new model for each new task is prohibitive in terms of computation and memory, due to the sheer number of parameters of deep CNNs. In this paper, we present a method to learn new-domains and tasks incrementally, building on prior knowledge from already learned tasks and without catastrophic forgetting. We do so by jointly parametrizing weights across layers using low-rank Tucker structure. The core is task agnostic while a set of task specific factors are learnt on each new domain. We show that leveraging tensor structure enables better performance than simply using matrix operations. Joint tensor modelling also naturally leverages correlations across different layers. Compared with previous methods which have focused on adapting each layer separately, our approach results in more compact representations for each new task/domain. We apply the proposed method to the 10 datasets of the Visual Decathlon Challenge and show that our method offers on average about 7.5x reduction in number of parameters and competitive performance in terms of both classification accuracy and Decathlon score.
Reliable Prediction Errors for Deep Neural Networks Using Test-Time Dropout
Authors: Isidro Cortes-Ciriano, Andreas Bender
Date: 2019-04-12 00:00:00
Tasks: Conformal Prediction, Decision Making, Drug Discovery, Prediction, Retrieval

While the use of deep learning in drug discovery is gaining increasing
attention, the lack of methods to compute reliable errors in prediction for
Neural Networks prevents their application to guide decision making in domains
where identifying unreliable predictions is essential, e.g. precision medicine.
Here, we present a framework to compute reliable errors in prediction for
Neural Networks using Test-Time Dropout and Conformal Prediction. Specifically,
the algorithm consists of training a single Neural Network using dropout, and
then applying it N times to both the validation and test sets, also employing
dropout in this step. Therefore, for each instance in the validation and test
sets an ensemble of predictions were generated. The residuals and absolute
errors in prediction for the validation set were then used to compute
prediction errors for test set instances using Conformal Prediction. We show
using 24 bioactivity data sets from ChEMBL 23 that dropout Conformal Predictors
are valid (i.e., the fraction of instances whose true value lies within the
predicted interval strongly correlates with the confidence level) and
efficient, as the predicted confidence intervals span a narrower set of values
than those computed with Conformal Predictors generated using Random Forest
(RF) models. Lastly, we show in retrospective virtual screening experiments
that dropout and RF-based Conformal Predictors lead to comparable retrieval
rates of active compounds. Overall, we propose a computationally efficient
framework (as only N extra forward passes are required in addition to training
a single network) to harness Test-Time Dropout and the Conformal Prediction
framework, and to thereby generate reliable prediction errors for deep Neural
Networks.
Boundary-Preserved Deep Denoising of the Stochastic Resonance Enhanced Multiphoton Images
Authors: Sheng-Yong Niu, Lun-Zhang Guo, Yue Li, Tzung-Dau Wang, Yu Tsao, Tzu-Ming Liu
Date: 2019-04-12 00:00:00
Tasks: Denoising, Texture Classification

As the rapid growth of high-speed and deep-tissue imaging in biomedical
research, it is urgent to find a robust and effective denoising method to
retain morphological features for further texture analysis and segmentation.
Conventional denoising filters and models can easily suppress perturbative
noises in high contrast images. However, for low photon budget multi-photon
images, high detector gain will not only boost signals, but also bring huge
background noises. In such stochastic resonance regime of imaging,
sub-threshold signals may be detectable with the help of noises. Therefore, a
denoising filter that can smartly remove noises without sacrificing the
important cellular features such as cell boundaries is highly desired. In this
paper, we propose a convolutional neural network based autoencoder method,
Fully Convolutional Deep Denoising Autoencoder (DDAE), to improve the quality
of Three-Photon Fluorescence (3PF) and Third Harmonic Generation (THG)
microscopy images. The average of the acquired 200 images of a given location
served as the low-noise answer for DDAE training. Compared with other widely
used denoising methods, our DDAE model shows better signal-to-noise ratio (26.6
and 29.9 for 3PF and THG, respectively), structure similarity (0.86 and 0.87
for 3PF and THG, respectively), and preservation of nuclear or cellular
boundaries.
Few-Shot Bayesian Imitation Learning with Logical Program Policies
Authors: Tom Silver, Kelsey R. Allen, Alex K. Lew, Leslie Pack Kaelbling, Josh Tenenbaum
Date: 2019-04-12 00:00:00
Tasks: Bayesian Inference, Imitation Learning, Program induction

Humans can learn many novel tasks from a very small number (1--5) of demonstrations, in stark contrast to the data requirements of nearly tabula rasa deep learning methods. We propose an expressive class of policies, a strong but general prior, and a learning algorithm that, together, can learn interesting policies from very few examples. We represent policies as logical combinations of programs drawn from a domain-specific language (DSL), define a prior over policies with a probabilistic grammar, and derive an approximate Bayesian inference algorithm to learn policies from demonstrations. In experiments, we study five strategy games played on a 2D grid with one shared DSL. After a few demonstrations of each game, the inferred policies generalize to new game instances that differ substantially from the demonstrations. Our policy learning is 20--1,000x more data efficient than convolutional and fully convolutional policy learning and many orders of magnitude more computationally efficient than vanilla program induction. We argue that the proposed method is an apt choice for tasks that have scarce training data and feature significant, structured variation between task instances.
Spatio-Temporal Deep Graph Infomax
Authors: Felix L. Opolka, Aaron Solomon, Cătălina Cangea, Petar Veličković, Pietro Liò, R. Devon Hjelm
Date: 2019-04-12 00:00:00
Tasks: Representation Learning, Traffic Prediction

Spatio-temporal graphs such as traffic networks or gene regulatory systems
present challenges for the existing deep learning methods due to the complexity
of structural changes over time. To address these issues, we introduce
Spatio-Temporal Deep Graph Infomax (STDGI)---a fully unsupervised node
representation learning approach based on mutual information maximization that
exploits both the temporal and spatial dynamics of the graph. Our model tackles
the challenging task of node-level regression by training embeddings to
maximize the mutual information between patches of the graph, at any given time
step, and between features of the central nodes of patches, in the future. We
demonstrate through experiments and qualitative studies that the learned
representations can successfully encode relevant information about the input
graph and improve the predictive performance of spatio-temporal auto-regressive
forecasting models.
Learning Optimal Decision Trees from Large Datasets
Authors: Florent Avellaneda
Date: 2019-04-12 00:00:00

Inferring a decision tree from a given dataset is one of the classic problems
in machine learning. This problem consists of buildings, from a labelled
dataset, a tree such that each node corresponds to a class and a path between
the tree root and a leaf corresponds to a conjunction of features to be
satisfied in this class. Following the principle of parsimony, we want to infer
a minimal tree consistent with the dataset. Unfortunately, inferring an optimal
decision tree is known to be NP-complete for several definitions of optimality.
Hence, the majority of existing approaches relies on heuristics, and as for the
few exact inference approaches, they do not work on large data sets. In this
paper, we propose a novel approach for inferring a decision tree of a minimum
depth based on the incremental generation of Boolean formula. The experimental
results indicate that it scales sufficiently well and the time it takes to run
grows slowly with the size of dataset.
Let's Play Again: Variability of Deep Reinforcement Learning Agents in Atari Environments
Authors: Kaleigh Clary, Emma Tosch, John Foley, David Jensen
Date: 2019-04-12 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reproducibility in reinforcement learning is challenging: uncontrolled
stochasticity from many sources, such as the learning algorithm, the learned
policy, and the environment itself have led researchers to report the
performance of learned agents using aggregate metrics of performance over
multiple random seeds for a single environment. Unfortunately, there are still
pernicious sources of variability in reinforcement learning agents that make
reporting common summary statistics an unsound metric for performance. Our
experiments demonstrate the variability of common agents used in the popular
OpenAI Baselines repository. We make the case for reporting post-training agent
performance as a distribution, rather than a point estimate.
Distributed Bandit Learning: Near-Optimal Regret with Efficient Communication
Authors: Yuanhao Wang, Jiachen Hu, Xiaoyu Chen, Li-Wei Wang
Date: 2019-04-12 00:00:00
Proceeding: ICLR 2020 1
Tasks: Multi-Armed Bandits

We study the problem of regret minimization for distributed bandits learning, in which $M$ agents work collaboratively to minimize their total regret under the coordination of a central server. Our goal is to design communication protocols with near-optimal regret and little communication cost, which is measured by the total amount of transmitted data. For distributed multi-armed bandits, we propose a protocol with near-optimal regret and only $O(M\log(MK))$ communication cost, where $K$ is the number of arms. The communication cost is independent of the time horizon $T$, has only logarithmic dependence on the number of arms, and matches the lower bound except for a logarithmic factor. For distributed $d$-dimensional linear bandits, we propose a protocol that achieves near-optimal regret and has communication cost of order $\tilde{O}(Md)$, which has only logarithmic dependence on $T$.
Revisit Lmser and its further development based on convolutional layers
Authors: Wenjing Huang, Shikui Tu, Lei Xu
Date: 2019-04-12 00:00:00

Proposed in 1991, Least Mean Square Error Reconstruction for self-organizing
network, shortly Lmser, was a further development of the traditional
auto-encoder (AE) by folding the architecture with respect to the central
coding layer and thus leading to the features of symmetric weights and neurons,
as well as jointly supervised and unsupervised learning. However, its
advantages were only demonstrated in a one-hidden-layer implementation due to
the lack of computing resources and big data at that time. In this paper, we
revisit Lmser from the perspective of deep learning, develop Lmser network
based on multiple convolutional layers, which is more suitable for
image-related tasks, and confirm several Lmser functions with preliminary
demonstrations on image recognition, reconstruction, association recall, and so
on. Experiments demonstrate that Lmser indeed works as indicated in the
original paper, and it has promising performance in various applications.
A Reference Vector based Many-Objective Evolutionary Algorithm with Feasibility-aware Adaptation
Authors: Mingde Zhao, Hongwei Ge, Kai Zhang, Yaqing Hou
Date: 2019-04-12 00:00:00
Tasks: Evolutionary Algorithms

The infeasible parts of the objective space in difficult many-objective
optimization problems cause trouble for evolutionary algorithms. This paper
proposes a reference vector based algorithm which uses two interacting engines
to adapt the reference vectors and to evolve the population towards the true
Pareto Front (PF) s.t. the reference vectors are always evenly distributed
within the current PF to provide appropriate guidance for selection. The
current PF is tracked by maintaining an archive of undominated individuals, and
adaptation of reference vectors is conducted with the help of another archive
that contains layers of reference vectors corresponding to different density.
Experimental results show the expected characteristics and competitive
performance of the proposed algorithm TEEA.