pwc-archive/papers-with-abstracts huggingface dataset:
Continuous-Time Flows for Efficient Inference and Density Estimation
Authors: Changyou Chen, Chunyuan Li, Liqun Chen, Wenlin Wang, Yunchen Pu, Lawrence Carin
Date: 2017-09-04 00:00:00
Proceeding: ICML 2018 7
Tasks: Density Estimation

Two fundamental problems in unsupervised learning are efficient inference for
latent-variable models and robust density estimation based on large amounts of
unlabeled data. Algorithms for the two tasks, such as normalizing flows and
generative adversarial networks (GANs), are often developed independently. In
this paper, we propose the concept of {\em continuous-time flows} (CTFs), a
family of diffusion-based methods that are able to asymptotically approach a
target distribution. Distinct from normalizing flows and GANs, CTFs can be
adopted to achieve the above two goals in one framework, with theoretical
guarantees. Our framework includes distilling knowledge from a CTF for
efficient inference, and learning an explicit energy-based distribution with
CTFs for density estimation. Both tasks rely on a new technique for
distribution matching within amortized learning. Experiments on various tasks
demonstrate promising performance of the proposed CTF framework, compared to
related techniques.
Joint Demosaicing and Denoising with Perceptual Optimization on a Generative Adversarial Network
Authors: Weishong Dong, Ming Yuan, Xin Li, Guangming Shi
Date: 2018-02-13 00:00:00
Tasks: Demosaicking, Denoising, Generative Adversarial Network, Joint Demosaicing and Denoising

Image demosaicing - one of the most important early stages in digital camera
pipelines - addressed the problem of reconstructing a full-resolution image
from so-called color-filter-arrays. Despite tremendous progress made in the
pase decade, a fundamental issue that remains to be addressed is how to assure
the visual quality of reconstructed images especially in the presence of noise
corruption. Inspired by recent advances in generative adversarial networks
(GAN), we present a novel deep learning approach toward joint demosaicing and
denoising (JDD) with perceptual optimization in order to ensure the visual
quality of reconstructed images. The key contributions of this work include: 1)
we have developed a GAN-based approach toward image demosacing in which a
discriminator network with both perceptual and adversarial loss functions are
used for quality assurance; 2) we propose to optimize the perceptual quality of
reconstructed images by the proposed GAN in an end-to-end manner. Such
end-to-end optimization of GAN is particularly effective for jointly exploiting
the gain brought by each modular component (e.g., residue learning in the
generative network and perceptual loss in the discriminator network). Our
extensive experimental results have shown convincingly improved performance
over existing state-of-the-art methods in terms of both subjective and
objective quality metrics with a comparable computational cost.
Predict and Constrain: Modeling Cardinality in Deep Structured Prediction
Authors: Nataly Brukhim, Amir Globerson
Date: 2018-02-13 00:00:00
Proceeding: ICML 2018 7
Tasks: BIG-bench Machine Learning, Deep Learning, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Prediction, Structured Prediction

Many machine learning problems require the prediction of multi-dimensional
labels. Such structured prediction models can benefit from modeling
dependencies between labels. Recently, several deep learning approaches to
structured prediction have been proposed. Here we focus on capturing
cardinality constraints in such models. Namely, constraining the number of
non-zero labels that the model outputs. Such constraints have proven very
useful in previous structured prediction approaches, but it is a challenge to
introduce them into a deep learning framework. Here we show how to do this via
a novel deep architecture. Our approach outperforms strong baselines, achieving
state-of-the-art results on multi-label classification benchmarks.
Training GANs with Optimism
Authors: Constantinos Daskalakis, Andrew Ilyas, Vasilis Syrgkanis, Haoyang Zeng
Date: 2017-10-31 00:00:00
Proceeding: ICLR 2018 1

We address the issue of limit cycling behavior in training Generative
Adversarial Networks and propose the use of Optimistic Mirror Decent (OMD) for
training Wasserstein GANs. Recent theoretical results have shown that
optimistic mirror decent (OMD) can enjoy faster regret rates in the context of
zero-sum games. WGANs is exactly a context of solving a zero-sum game with
simultaneous no-regret dynamics. Moreover, we show that optimistic mirror
decent addresses the limit cycling problem in training WGANs. We formally show
that in the case of bi-linear zero-sum games the last iterate of OMD dynamics
converges to an equilibrium, in contrast to GD dynamics which are bound to
cycle. We also portray the huge qualitative difference between GD and OMD
dynamics with toy examples, even when GD is modified with many adaptations
proposed in the recent literature, such as gradient penalty or momentum. We
apply OMD WGAN training to a bioinformatics problem of generating DNA
sequences. We observe that models trained with OMD achieve consistently smaller
KL divergence with respect to the true underlying distribution, than models
trained with GD variants. Finally, we introduce a new algorithm, Optimistic
Adam, which is an optimistic variant of Adam. We apply it to WGAN training on
CIFAR10 and observe improved performance in terms of inception score as
compared to Adam.
Learning Registered Point Processes from Idiosyncratic Observations
Authors: Hongteng Xu, Lawrence Carin, Hongyuan Zha
Date: 2017-10-03 00:00:00
Proceeding: ICML 2018 7
Tasks: Point Processes

A parametric point process model is developed, with modeling based on the
assumption that sequential observations often share latent phenomena, while
also possessing idiosyncratic effects. An alternating optimization method is
proposed to learn a "registered" point process that accounts for shared
structure, as well as "warping" functions that characterize idiosyncratic
aspects of each observed sequence. Under reasonable constraints, in each
iteration we update the sample-specific warping functions by solving a set of
constrained nonlinear programming problems in parallel, and update the model by
maximum likelihood estimation. The justifiability, complexity and robustness of
the proposed method are investigated in detail, and the influence of sequence
stitching on the learning results is examined empirically. Experiments on both
synthetic and real-world data demonstrate that the method yields explainable
point process models, achieving encouraging results compared to
state-of-the-art methods.
Hadamard Response: Estimating Distributions Privately, Efficiently, and with Little Communication
Authors: Jayadev Acharya, Ziteng Sun, Huanyu Zhang
Date: 2018-02-13 00:00:00

We study the problem of estimating $k$-ary distributions under
$\varepsilon$-local differential privacy. $n$ samples are distributed across
users who send privatized versions of their sample to a central server. All
previously known sample optimal algorithms require linear (in $k$)
communication from each user in the high privacy regime $(\varepsilon=O(1))$,
and run in time that grows as $n\cdot k$, which can be prohibitive for large
domain size $k$.
  We propose Hadamard Response (HR}, a local privatization scheme that requires
no shared randomness and is symmetric with respect to the users. Our scheme has
order optimal sample complexity for all $\varepsilon$, a communication of at
most $\log k+2$ bits per user, and nearly linear running time of $\tilde{O}(n +
k)$.
  Our encoding and decoding are based on Hadamard matrices, and are simple to
implement. The statistical performance relies on the coding theoretic aspects
of Hadamard matrices, ie, the large Hamming distance between the rows. An
efficient implementation of the algorithm using the Fast Walsh-Hadamard
transform gives the computational gains.
  We compare our approach with Randomized Response (RR), RAPPOR, and
subset-selection mechanisms (SS), both theoretically, and experimentally. For
$k=10000$, our algorithm runs about 100x faster than SS, and RAPPOR.
Deep Adversarial Attention Alignment for Unsupervised Domain Adaptation: the Benefit of Target Expectation Maximization
Authors: Guoliang Kang, Liang Zheng, Yan Yan, Yi Yang
Date: 2018-01-30 00:00:00
Proceeding: ECCV 2018 9
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

In this paper, we make two contributions to unsupervised domain adaptation
(UDA) using the convolutional neural network (CNN). First, our approach
transfers knowledge in all the convolutional layers through attention
alignment. Most previous methods align high-level representations, e.g.,
activations of the fully connected (FC) layers. In these methods, however, the
convolutional layers which underpin critical low-level domain knowledge cannot
be updated directly towards reducing domain discrepancy. Specifically, we
assume that the discriminative regions in an image are relatively invariant to
image style changes. Based on this assumption, we propose an attention
alignment scheme on all the target convolutional layers to uncover the
knowledge shared by the source domain. Second, we estimate the posterior label
distribution of the unlabeled data for target network training. Previous
methods, which iteratively update the pseudo labels by the target network and
refine the target network by the updated pseudo labels, are vulnerable to label
estimation errors. Instead, our approach uses category distribution to
calculate the cross-entropy loss for training, thereby ameliorating the error
accumulation of the estimated labels. The two contributions allow our approach
to outperform the state-of-the-art methods by +2.6% on the Office-31 dataset.
Fully Convolutional Architectures for Multi-Class Segmentation in Chest Radiographs
Authors: Alexey A. Novikov, Dimitrios Lenis, David Major, Jiri Hladůvka, Maria Wimmer, Katja Bühler
Date: 2017-01-30 00:00:00
Tasks: image-classification, Image Classification

The success of deep convolutional neural networks on image classification and
recognition tasks has led to new applications in very diversified contexts,
including the field of medical imaging. In this paper we investigate and
propose neural network architectures for automated multi-class segmentation of
anatomical organs in chest radiographs, namely for lungs, clavicles and heart.
We address several open challenges including model overfitting, reducing number
of parameters and handling of severely imbalanced data in CXR by fusing recent
concepts in convolutional networks and adapting them to the segmentation
problem task in CXR. We demonstrate that our architecture combining delayed
subsampling, exponential linear units, highly restrictive regularization and a
large number of high resolution low level abstract features outperforms
state-of-the-art methods on all considered organs, as well as the human
observer on lungs and heart. The models use a multi-class configuration with
three target classes and are trained and tested on the publicly available JSRT
database, consisting of 247 X-ray images the ground-truth masks for which are
available in the SCR database. Our best performing model, trained with the loss
function based on the Dice coefficient, reached mean Jaccard overlap scores of
95.0\% for lungs, 86.8\% for clavicles and 88.2\% for heart. This architecture
outperformed the human observer results for lungs and heart.
Learning to Search with MCTSnets
Authors: Arthur Guez, Théophane Weber, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals, Daan Wierstra, Rémi Munos, David Silver
Date: 2018-02-13 00:00:00
Proceeding: ICML 2018 7
Tasks: Sokoban

Planning problems are among the most important and well-studied problems in
artificial intelligence. They are most typically solved by tree search
algorithms that simulate ahead into the future, evaluate future states, and
back-up those evaluations to the root of a search tree. Among these algorithms,
Monte-Carlo tree search (MCTS) is one of the most general, powerful and widely
used. A typical implementation of MCTS uses cleverly designed rules, optimized
to the particular characteristics of the domain. These rules control where the
simulation traverses, what to evaluate in the states that are reached, and how
to back-up those evaluations. In this paper we instead learn where, what and
how to search. Our architecture, which we call an MCTSnet, incorporates
simulation-based search inside a neural network, by expanding, evaluating and
backing-up a vector embedding. The parameters of the network are trained
end-to-end using gradient-based optimisation. When applied to small searches in
the well known planning problem Sokoban, the learned search algorithm
significantly outperformed MCTS baselines.
Generative ScatterNet Hybrid Deep Learning (G-SHDL) Network with Structural Priors for Semantic Image Segmentation
Authors: Amarjot Singh, Nick Kingsbury
Date: 2018-02-09 00:00:00
Tasks: General Classification, Image Segmentation, Semantic Segmentation

This paper proposes a generative ScatterNet hybrid deep learning (G-SHDL)
network for semantic image segmentation. The proposed generative architecture
is able to train rapidly from relatively small labeled datasets using the
introduced structural priors. In addition, the number of filters in each layer
of the architecture is optimized resulting in a computationally efficient
architecture. The G-SHDL network produces state-of-the-art classification
performance against unsupervised and semi-supervised learning on two image
datasets. Advantages of the G-SHDL network over supervised methods are
demonstrated with experiments performed on training datasets of reduced size.
BIRNet: Brain Image Registration Using Dual-Supervised Fully Convolutional Networks
Authors: Jingfan Fan, Xiaohuan Cao, Pew-Thian Yap, Dinggang Shen
Date: 2018-02-13 00:00:00
Tasks: Image Registration

In this paper, we propose a deep learning approach for image registration by
predicting deformation from image appearance. Since obtaining ground-truth
deformation fields for training can be challenging, we design a fully
convolutional network that is subject to dual-guidance: (1) Coarse guidance
using deformation fields obtained by an existing registration method; and (2)
Fine guidance using image similarity. The latter guidance helps avoid overly
relying on the supervision from the training deformation fields, which could be
inaccurate. For effective training, we further improve the deep convolutional
network with gap filling, hierarchical loss, and multi-source strategies.
Experiments on a variety of datasets show promising registration accuracy and
efficiency compared with state-of-the-art methods.
Unsupervised Evaluation and Weighted Aggregation of Ranked Predictions
Authors: Mehmet Eren Ahsen, Robert Vogel, Gustavo Stolovitzky
Date: 2018-02-13 00:00:00
Tasks: Binary Classification, Ensemble Learning, General Classification

Learning algorithms that aggregate predictions from an ensemble of diverse
base classifiers consistently outperform individual methods. Many of these
strategies have been developed in a supervised setting, where the accuracy of
each base classifier can be empirically measured and this information is
incorporated in the training process. However, the reliance on labeled data
precludes the application of ensemble methods to many real world problems where
labeled data has not been curated. To this end we developed a new theoretical
framework for binary classification, the Strategy for Unsupervised Multiple
Method Aggregation (SUMMA), to estimate the performances of base classifiers
and an optimal strategy for ensemble learning from unlabeled data.
Examining the Tip of the Iceberg: A Data Set for Idiom Translation
Authors: Marzieh Fadaee, Arianna Bisazza, Christof Monz
Date: 2018-02-13 00:00:00
Proceeding: LREC 2018 5
Tasks: Machine Translation, NMT, Translation

Neural Machine Translation (NMT) has been widely used in recent years with
significant improvements for many language pairs. Although state-of-the-art NMT
systems are generating progressively better translations, idiom translation
remains one of the open challenges in this field. Idioms, a category of
multiword expressions, are an interesting language phenomenon where the overall
meaning of the expression cannot be composed from the meanings of its parts. A
first important challenge is the lack of dedicated data sets for learning and
evaluating idiom translation. In this paper we address this problem by creating
the first large-scale data set for idiom translation. Our data set is
automatically extracted from a widely used German-English translation corpus
and includes, for each language direction, a targeted evaluation set where all
sentences contain idioms and a regular training corpus where sentences
including idioms are marked. We release this data set and use it to perform
preliminary NMT experiments as the first step towards better idiom translation.
Training and Inference with Integers in Deep Neural Networks
Authors: Shuang Wu, Guoqi Li, Feng Chen, Luping Shi
Date: 2018-02-13 00:00:00
Proceeding: ICLR 2018 1
Tasks: Continual Learning

Researches on deep neural networks with discrete parameters and their
deployment in embedded systems have been active and promising topics. Although
previous works have successfully reduced precision in inference, transferring
both training and inference processes to low-bitwidth integers has not been
demonstrated simultaneously. In this work, we develop a new method termed as
"WAGE" to discretize both training and inference, where weights (W),
activations (A), gradients (G) and errors (E) among layers are shifted and
linearly constrained to low-bitwidth integers. To perform pure discrete
dataflow for fixed-point devices, we further replace batch normalization by a
constant scaling layer and simplify other components that are arduous for
integer implementation. Improved accuracies can be obtained on multiple
datasets, which indicates that WAGE somehow acts as a type of regularization.
Empirically, we demonstrate the potential to deploy training in hardware
systems such as integer-based deep learning accelerators and neuromorphic chips
with comparable accuracy and higher energy efficiency, which is crucial to
future AI applications in variable scenarios with transfer and continual
learning demands.
Variable Selection and Task Grouping for Multi-Task Learning
Authors: Jun-Yong Jeong, Chi-Hyuck Jun
Date: 2018-02-13 00:00:00
Tasks: Multi-Task Learning, Variable Selection

We consider multi-task learning, which simultaneously learns related
prediction tasks, to improve generalization performance. We factorize a
coefficient matrix as the product of two matrices based on a low-rank
assumption. These matrices have sparsities to simultaneously perform variable
selection and learn and overlapping group structure among the tasks. The
resulting bi-convex objective function is minimized by alternating optimization
where sub-problems are solved using alternating direction method of multipliers
and accelerated proximal gradient descent. Moreover, we provide the performance
bound of the proposed method. The effectiveness of the proposed method is
validated for both synthetic and real-world datasets.
Attention based Sentence Extraction from Scientific Articles using Pseudo-Labeled data
Authors: Parth Mehta, Gaurav Arora, Prasenjit Majumder
Date: 2018-02-13 00:00:00
Tasks: Articles, Sentence, Topic Models

In this work, we present a weakly supervised sentence extraction technique
for identifying important sentences in scientific papers that are worthy of
inclusion in the abstract. We propose a new attention based deep learning
architecture that jointly learns to identify important content, as well as the
cue phrases that are indicative of summary worthy sentences. We propose a new
context embedding technique for determining the focus of a given paper using
topic models and use it jointly with an LSTM based sequence encoder to learn
attention weights across the sentence words. We use a collection of articles
publicly available through ACL anthology for our experiments. Our system
achieves a performance that is better, in terms of several ROUGE metrics, as
compared to several state of art extractive techniques. It also generates more
coherent summaries and preserves the overall structure of the document.
Extracting Sub-Exposure Images from a Single Capture Through Fourier-based Optical Modulation
Authors: Shah Rez Khan, Martin Feldman, Bahadir K. Gunturk
Date: 2016-12-26 00:00:00
Tasks: compressed sensing

Through pixel-wise optical coding of images during exposure time, it is
possible to extract sub-exposure images from a single capture. Such a
capability can be used for different purposes, including high-speed imaging,
high-dynamic-range imaging and compressed sensing. In this paper, we
demonstrate a sub-exposure image extraction method, where the exposure coding
pattern is inspired from frequency division multiplexing idea of communication
systems. The coding masks modulate sub-exposure images in such a way that they
are placed in non-overlapping regions in Fourier domain. The sub-exposure image
extraction process involves digital filtering of the captured signal with
proper band-pass filters. The prototype imaging system incorporates a Liquid
Crystal over Silicon (LCoS) based spatial light modulator synchronized with a
camera for pixel-wise exposure coding.
Weakly supervised collective feature learning from curated media
Authors: Yusuke Mukuta, Akisato Kimura, David B Adrian, Zoubin Ghahramani
Date: 2018-02-13 00:00:00
Tasks: Link Prediction, TAG

The current state-of-the-art in feature learning relies on the supervised
learning of large-scale datasets consisting of target content items and their
respective category labels. However, constructing such large-scale
fully-labeled datasets generally requires painstaking manual effort. One
possible solution to this problem is to employ community contributed text tags
as weak labels, however, the concepts underlying a single text tag strongly
depends on the users. We instead present a new paradigm for learning
discriminative features by making full use of the human curation process on
social networking services (SNSs). During the process of content curation, SNS
users collect content items manually from various sources and group them by
context, all for their own benefit. Due to the nature of this process, we can
assume that (1) content items in the same group share the same semantic concept
and (2) groups sharing the same images might have related semantic concepts.
Through these insights, we can define human curated groups as weak labels from
which our proposed framework can learn discriminative features as a
representation in the space of semantic concepts the users intended when
creating the groups. We show that this feature learning can be formulated as a
problem of link prediction for a bipartite graph whose nodes corresponds to
content items and human curated groups, and propose a novel method for feature
learning based on sparse coding or network fine-tuning.