pwc-archive/papers-with-abstracts huggingface dataset:
A Spectral Approach to Gradient Estimation for Implicit Distributions
Authors: Jiaxin Shi, Shengyang Sun, Jun Zhu
Date: 2018-06-07 00:00:00
Proceeding: ICML 2018 7
Tasks: Variational Inference

Recently there have been increasing interests in learning and inference with
implicit distributions (i.e., distributions without tractable densities). To
this end, we develop a gradient estimator for implicit distributions based on
Stein's identity and a spectral decomposition of kernel operators, where the
eigenfunctions are approximated by the Nystr\"om method. Unlike the previous
works that only provide estimates at the sample points, our approach directly
estimates the gradient function, thus allows for a simple and principled
out-of-sample extension. We provide theoretical results on the error bound of
the estimator and discuss the bias-variance tradeoff in practice. The
effectiveness of our method is demonstrated by applications to gradient-free
Hamiltonian Monte Carlo and variational inference with implicit distributions.
Finally, we discuss the intuition behind the estimator by drawing connections
between the Nystr\"om method and kernel PCA, which indicates that the estimator
can automatically adapt to the geometry of the underlying distribution.
Multimodal Relational Tensor Network for Sentiment and Emotion Classification
Authors: Saurav Sahay, Shachi H. Kumar, Rui Xia, Jonathan Huang, Lama Nachman
Date: 2018-06-07 00:00:00
Proceeding: WS 2018 7
Tasks: Classification, Emotion Classification, Emotion Recognition, General Classification, Sentiment Analysis, Sentiment Classification

Understanding Affect from video segments has brought researchers from the
language, audio and video domains together. Most of the current multimodal
research in this area deals with various techniques to fuse the modalities, and
mostly treat the segments of a video independently. Motivated by the work of
(Zadeh et al., 2017) and (Poria et al., 2017), we present our architecture,
Relational Tensor Network, where we use the inter-modal interactions within a
segment (intra-segment) and also consider the sequence of segments in a video
to model the inter-segment inter-modal interactions. We also generate rich
representations of text and audio modalities by leveraging richer audio and
linguistic context alongwith fusing fine-grained knowledge based polarity
scores from text. We present the results of our model on CMU-MOSEI dataset and
show that our model outperforms many baselines and state of the art methods for
sentiment classification and emotion recognition.
Feature selection in functional data classification with recursive maxima hunting
Authors: José L. Torrecilla, Alberto Suárez
Date: 2018-06-07 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Dimensionality Reduction, feature selection, General Classification, Variable Selection

Dimensionality reduction is one of the key issues in the design of effective
machine learning methods for automatic induction. In this work, we introduce
recursive maxima hunting (RMH) for variable selection in classification
problems with functional data. In this context, variable selection techniques
are especially attractive because they reduce the dimensionality, facilitate
the interpretation and can improve the accuracy of the predictive models. The
method, which is a recursive extension of maxima hunting (MH), performs
variable selection by identifying the maxima of a relevance function, which
measures the strength of the correlation of the predictor functional variable
with the class label. At each stage, the information associated with the
selected variable is removed by subtracting the conditional expectation of the
process. The results of an extensive empirical evaluation are used to
illustrate that, in the problems investigated, RMH has comparable or higher
predictive accuracy than the standard dimensionality reduction techniques, such
as PCA and PLS, and state-of-the-art feature selection methods for functional
data, such as maxima hunting.
RadialGAN: Leveraging multiple datasets to improve target-specific predictive models using Generative Adversarial Networks
Authors: Jinsung Yoon, James Jordon, Mihaela van der Schaar
Date: 2018-02-18 00:00:00
Proceeding: ICML 2018 7

Training complex machine learning models for prediction often requires a
large amount of data that is not always readily available. Leveraging these
external datasets from related but different sources is therefore an important
task if good predictive models are to be built for deployment in settings where
data can be rare. In this paper we propose a novel approach to the problem in
which we use multiple GAN architectures to learn to translate from one dataset
to another, thereby allowing us to effectively enlarge the target dataset, and
therefore learn better predictive models than if we simply used the target
dataset. We show the utility of such an approach, demonstrating that our method
improves the prediction performance on the target domain over using just the
target dataset and also show that our framework outperforms several other
benchmarks on a collection of real-world medical datasets.
Location Name Extraction from Targeted Text Streams using Gazetteer-based Statistical Language Models
Authors: Hussein S. Al-Olimat, Krishnaprasad Thirunarayan, Valerie Shalin, Amit Sheth
Date: 2017-08-10 00:00:00
Proceeding: COLING 2018 8
Tasks: Language Modeling, Language Modelling

Extracting location names from informal and unstructured social media data requires the identification of referent boundaries and partitioning compound names. Variability, particularly systematic variability in location names (Carroll, 1983), challenges the identification task. Some of this variability can be anticipated as operations within a statistical language model, in this case drawn from gazetteers such as OpenStreetMap (OSM), Geonames, and DBpedia. This permits evaluation of an observed n-gram in Twitter targeted text as a legitimate location name variant from the same location-context. Using n-gram statistics and location-related dictionaries, our Location Name Extraction tool (LNEx) handles abbreviations and automatically filters and augments the location names in gazetteers (handling name contractions and auxiliary contents) to help detect the boundaries of multi-word location names and thereby delimit them in texts. We evaluated our approach on 4,500 event-specific tweets from three targeted streams to compare the performance of LNEx against that of ten state-of-the-art taggers that rely on standard semantic, syntactic and/or orthographic features. LNEx improved the average F-Score by 33-179%, outperforming all taggers. Further, LNEx is capable of stream processing.
GAIN: Missing Data Imputation using Generative Adversarial Nets
Authors: Jinsung Yoon, James Jordon, Mihaela van der Schaar
Date: 2018-06-07 00:00:00
Proceeding: ICML 2018 7
Tasks: Imputation, Multivariate Time Series Imputation

We propose a novel method for imputing missing data by adapting the
well-known Generative Adversarial Nets (GAN) framework. Accordingly, we call
our method Generative Adversarial Imputation Nets (GAIN). The generator (G)
observes some components of a real data vector, imputes the missing components
conditioned on what is actually observed, and outputs a completed vector. The
discriminator (D) then takes a completed vector and attempts to determine which
components were actually observed and which were imputed. To ensure that D
forces G to learn the desired distribution, we provide D with some additional
information in the form of a hint vector. The hint reveals to D partial
information about the missingness of the original sample, which is used by D to
focus its attention on the imputation quality of particular components. This
hint ensures that G does in fact learn to generate according to the true data
distribution. We tested our method on various datasets and found that GAIN
significantly outperforms state-of-the-art imputation methods.
Non-Local Recurrent Network for Image Restoration
Authors: Ding Liu, Bihan Wen, Yuchen Fan, Chen Change Loy, Thomas S. Huang
Date: 2018-06-07 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Denoising, Feature Correlation, Image Denoising, Image Restoration, Image Super-Resolution, Super-Resolution

Many classic methods have shown non-local self-similarity in natural images
to be an effective prior for image restoration. However, it remains unclear and
challenging to make use of this intrinsic property via deep networks. In this
paper, we propose a non-local recurrent network (NLRN) as the first attempt to
incorporate non-local operations into a recurrent neural network (RNN) for
image restoration. The main contributions of this work are: (1) Unlike existing
methods that measure self-similarity in an isolated manner, the proposed
non-local module can be flexibly integrated into existing deep networks for
end-to-end training to capture deep feature correlation between each location
and its neighborhood. (2) We fully employ the RNN structure for its parameter
efficiency and allow deep feature correlation to be propagated along adjacent
recurrent states. This new design boosts robustness against inaccurate
correlation estimation due to severely degraded images. (3) We show that it is
essential to maintain a confined neighborhood for computing deep feature
correlation given degraded images. This is in contrast to existing practice
that deploys the whole image. Extensive experiments on both image denoising and
super-resolution tasks are conducted. Thanks to the recurrent non-local
operations and correlation propagation, the proposed NLRN achieves superior
results to state-of-the-art methods with much fewer parameters.
Color Sails: Discrete-Continuous Palettes for Deep Color Exploration
Authors: Maria Shugrina, Amlan Kar, Karan Singh, Sanja Fidler
Date: 2018-06-07 00:00:00

We present color sails, a discrete-continuous color gamut representation that
extends the color gradient analogy to three dimensions and allows interactive
control of the color blending behavior. Our representation models a wide
variety of color distributions in a compact manner, and lends itself to
applications such as color exploration for graphic design, illustration and
similar fields. We propose a Neural Network that can fit a color sail to any
image. Then, the user can adjust color sail parameters to change the base
colors, their blending behavior and the number of colors, exploring a wide
range of options for the original design. In addition, we propose a Deep
Learning model that learns to automatically segment an image into
color-compatible alpha masks, each equipped with its own color sail. This
allows targeted color exploration by either editing their corresponding color
sails or using standard software packages. Our model is trained on a custom
diverse dataset of art and design. We provide both quantitative evaluations,
and a user study, demonstrating the effectiveness of color sail interaction.
Interactive demos are available at www.colorsails.com.
Stein Variational Message Passing for Continuous Graphical Models
Authors: Dilin Wang, Zhe Zeng, Qiang Liu
Date: 2017-11-20 00:00:00
Proceeding: ICML 2018 7

We propose a novel distributed inference algorithm for continuous graphical
models, by extending Stein variational gradient descent (SVGD) to leverage the
Markov dependency structure of the distribution of interest. Our approach
combines SVGD with a set of structured local kernel functions defined on the
Markov blanket of each node, which alleviates the curse of high dimensionality
and simultaneously yields a distributed algorithm for decentralized inference
tasks. We justify our method with theoretical analysis and show that the use of
local kernels can be viewed as a new type of localized approximation that
matches the target distribution on the conditional distributions of each node
over its Markov blanket. Our empirical results show that our method outperforms
a variety of baselines including standard MCMC and particle message passing
methods.
Adversarial Label Learning
Authors: Chidubem Arachie, Bert Huang
Date: 2018-05-22 00:00:00
Tasks: Weakly-supervised Learning

We consider the task of training classifiers without labels. We propose a
weakly supervised method---adversarial label learning---that trains classifiers
to perform well against an adversary that chooses labels for training data. The
weak supervision constrains what labels the adversary can choose. The method
therefore minimizes an upper bound of the classifier's error rate using
projected primal-dual subgradient descent. Minimizing this bound protects
against bias and dependencies in the weak supervision. Experiments on three
real datasets show that our method can train without labels and outperforms
other approaches for weakly supervised learning.
Efficient Discovery of Heterogeneous Quantile Treatment Effects in Randomized Experiments via Anomalous Pattern Detection
Authors: Edward McFowland III, Sriram Somanchi, Daniel B. Neill
Date: 2018-03-24 00:00:00

In the recent literature on estimating heterogeneous treatment effects, each proposed method makes its own set of restrictive assumptions about the intervention's effects and which subpopulations to explicitly estimate. Moreover, the majority of the literature provides no mechanism to identify which subpopulations are the most affected--beyond manual inspection--and provides little guarantee on the correctness of the identified subpopulations. Therefore, we propose Treatment Effect Subset Scan (TESS), a new method for discovering which subpopulation in a randomized experiment is most significantly affected by a treatment. We frame this challenge as a pattern detection problem where we efficiently maximize a nonparametric scan statistic (a measure of the conditional quantile treatment effect) over subpopulations. Furthermore, we identify the subpopulation which experiences the largest distributional change as a result of the intervention, while making minimal assumptions about the intervention's effects or the underlying data generating process. In addition to the algorithm, we demonstrate that under the sharp null hypothesis of no treatment effect, the asymptotic Type I and II error can be controlled, and provide sufficient conditions for detection consistency--i.e., exact identification of the affected subpopulation. Finally, we validate the efficacy of the method by discovering heterogeneous treatment effects in simulations and in real-world data from a well-known program evaluation study.
Towards Fast Computation of Certified Robustness for ReLU Networks
Authors: Tsui-Wei Weng, huan zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Duane Boning, Inderjit S. Dhillon, Luca Daniel
Date: 2018-04-25 00:00:00
Proceeding: ICML 2018 7
Tasks: CPU

Verifying the robustness property of a general Rectified Linear Unit (ReLU)
network is an NP-complete problem [Katz, Barrett, Dill, Julian and Kochenderfer
CAV17]. Although finding the exact minimum adversarial distortion is hard,
giving a certified lower bound of the minimum distortion is possible. Current
available methods of computing such a bound are either time-consuming or
delivering low quality bounds that are too loose to be useful. In this paper,
we exploit the special structure of ReLU networks and provide two
computationally efficient algorithms Fast-Lin and Fast-Lip that are able to
certify non-trivial lower bounds of minimum distortions, by bounding the ReLU
units with appropriate linear functions Fast-Lin, or by bounding the local
Lipschitz constant Fast-Lip. Experiments show that (1) our proposed methods
deliver bounds close to (the gap is 2-3X) exact minimum distortion found by
Reluplex in small MNIST networks while our algorithms are more than 10,000
times faster; (2) our methods deliver similar quality of bounds (the gap is
within 35% and usually around 10%; sometimes our bounds are even better) for
larger networks compared to the methods based on solving linear programming
problems but our algorithms are 33-14,000 times faster; (3) our method is
capable of solving large MNIST and CIFAR networks up to 7 layers with more than
10,000 neurons within tens of seconds on a single CPU core.
  In addition, we show that, in fact, there is no polynomial time algorithm
that can approximately find the minimum $\ell_1$ adversarial distortion of a
ReLU network with a $0.99\ln n$ approximation ratio unless
$\mathsf{NP}$=$\mathsf{P}$, where $n$ is the number of neurons in the network.
Fast Decoding in Sequence Models using Discrete Latent Variables
Authors: Łukasz Kaiser, Aurko Roy, Ashish Vaswani, Niki Parmar, Samy Bengio, Jakob Uszkoreit, Noam Shazeer
Date: 2018-03-09 00:00:00
Proceeding: ICML 2018 7
Tasks: Machine Translation, Translation

Autoregressive sequence models based on deep neural networks, such as RNNs,
Wavenet and the Transformer attain state-of-the-art results on many tasks.
However, they are difficult to parallelize and are thus slow at processing long
sequences. RNNs lack parallelism both during training and decoding, while
architectures like WaveNet and Transformer are much more parallelizable during
training, yet still operate sequentially during decoding.
  Inspired by [arxiv:1711.00937], we present a method to extend sequence models
using discrete latent variables that makes decoding much more parallelizable.
We first auto-encode the target sequence into a shorter sequence of discrete
latent variables, which at inference time is generated autoregressively, and
finally decode the output sequence from this shorter latent sequence in
parallel. To this end, we introduce a novel method for constructing a sequence
of discrete latent variables and compare it with previously introduced methods.
Finally, we evaluate our model end-to-end on the task of neural machine
translation, where it is an order of magnitude faster at decoding than
comparable autoregressive models. While lower in BLEU than purely
autoregressive models, our model achieves higher scores than previously
proposed non-autoregressive translation models.
From Nodes to Networks: Evolving Recurrent Neural Networks
Authors: Aditya Rawal, Risto Miikkulainen
Date: 2018-03-12 00:00:00
Tasks: Language Modeling, Language Modelling, Machine Translation, Reinforcement Learning, speech-recognition, Speech Recognition, Translation

Gated recurrent networks such as those composed of Long Short-Term Memory
(LSTM) nodes have recently been used to improve state of the art in many
sequential processing tasks such as speech recognition and machine translation.
However, the basic structure of the LSTM node is essentially the same as when
it was first conceived 25 years ago. Recently, evolutionary and reinforcement
learning mechanisms have been employed to create new variations of this
structure. This paper proposes a new method, evolution of a tree-based encoding
of the gated memory nodes, and shows that it makes it possible to explore new
variations more effectively than other methods. The method discovers nodes with
multiple recurrent paths and multiple memory cells, which lead to significant
improvement in the standard language modeling benchmark task. The paper also
shows how the search process can be speeded up by training an LSTM network to
estimate performance of candidate structures, and by encouraging exploration of
novel solutions. Thus, evolutionary design of complex neural network structures
promises to improve performance of deep learning architectures beyond human
ability to do so.
CapsGAN: Using Dynamic Routing for Generative Adversarial Networks
Authors: Raeid Saqur, Sal Vivona
Date: 2018-06-07 00:00:00
Tasks: Image Generation, Rotated MNIST

In this paper, we propose a novel technique for generating images in the 3D
domain from images with high degree of geometrical transformations. By
coalescing two popular concurrent methods that have seen rapid ascension to the
machine learning zeitgeist in recent years: GANs (Goodfellow et. al.) and
Capsule networks (Sabour, Hinton et. al.) - we present: \textbf{CapsGAN}. We
show that CapsGAN performs better than or equal to traditional CNN based GANs
in generating images with high geometric transformations using rotated MNIST.
In the process, we also show the efficacy of using capsules architecture in the
GANs domain. Furthermore, we tackle the Gordian Knot in training GANs - the
performance control and training stability by experimenting with using
Wasserstein distance (gradient clipping, penalty) and Spectral Normalization.
The experimental findings of this paper should propel the application of
capsules and GANs in the still exciting and nascent domain of 3D image
generation, and plausibly video (frame) generation.
Accelerating Natural Gradient with Higher-Order Invariance
Authors: Yang Song, Jiaming Song, Stefano Ermon
Date: 2018-03-04 00:00:00
Proceeding: ICML 2018 7
Tasks: Reinforcement Learning

An appealing property of the natural gradient is that it is invariant to
arbitrary differentiable reparameterizations of the model. However, this
invariance property requires infinitesimal steps and is lost in practical
implementations with small but finite step sizes. In this paper, we study
invariance properties from a combined perspective of Riemannian geometry and
numerical differential equation solving. We define the order of invariance of a
numerical method to be its convergence order to an invariant solution. We
propose to use higher-order integrators and geodesic corrections to obtain more
invariant optimization trajectories. We prove the numerical convergence
properties of geodesic corrected updates and show that they can be as
computationally efficient as plain natural gradient. Experimentally, we
demonstrate that invariance leads to faster optimization and our techniques
improve on traditional natural gradient in deep neural network training and
natural policy gradient for reinforcement learning.
Is preprocessing of text really worth your time for online comment classification?
Authors: Fahim Mohammad
Date: 2018-06-07 00:00:00
Tasks: General Classification, Toxic Comment Classification

A large proportion of online comments present on public domains are
constructive, however a significant proportion are toxic in nature. The
comments contain lot of typos which increases the number of features manifold,
making the ML model difficult to train. Considering the fact that the data
scientists spend approximately 80% of their time in collecting, cleaning and
organizing their data [1], we explored how much effort should we invest in the
preprocessing (transformation) of raw comments before feeding it to the
state-of-the-art classification models. With the help of four models on Jigsaw
toxic comment classification data, we demonstrated that the training of model
without any transformation produce relatively decent model. Applying even basic
transformations, in some cases, lead to worse performance and should be applied
with caution.
Copy Move Forgery using Hus Invariant Moments and Log Polar Transformations
Authors: Tejas K, Swathi C, Rajesh Kumar M
Date: 2018-06-07 00:00:00

With the increase in interchange of data, there is a growing necessity of
security. Considering the volumes of digital data that is transmitted, they are
in need to be secure. Among the many forms of tampering possible, one
widespread technique is Copy Move Forgery CMF. This forgery occurs when parts
of the image are copied and duplicated elsewhere in the same image. There exist
a number of algorithms to detect such a forgery in which the primary step
involved is feature extraction. The feature extraction techniques employed must
have lesser time and space complexity involved for an efficient and faster
processing of media. Also, majority of the existing state of art techniques
often tend to falsely match similar genuine objects as copy move forged during
the detection process. To tackle these problems, the paper proposes a novel
algorithm that recognizes a unique approach of using Hus Invariant Moments and
Log polar Transformations to reduce feature vector dimension to one feature per
block simultaneously detecting CMF among genuine similar objects in an image.
The qualitative and quantitative results obtained demonstrate the effectiveness
of this algorithm.