pwc-archive/papers-with-abstracts huggingface dataset:
Post-Regularization Inference for Time-Varying Nonparanormal Graphical Models
Authors: Junwei Lu, Mladen Kolar, Han Liu
Date: 2015-12-28 00:00:00
Tasks: Model Selection

We propose a novel class of time-varying nonparanormal graphical models,
which allows us to model high dimensional heavy-tailed systems and the
evolution of their latent network structures. Under this model, we develop
statistical tests for presence of edges both locally at a fixed index value and
globally over a range of values. The tests are developed for a high-dimensional
regime, are robust to model selection mistakes and do not require commonly
assumed minimum signal strength. The testing procedures are based on a high
dimensional, debiasing-free moment estimator, which uses a novel kernel
smoothed Kendall's tau correlation matrix as an input statistic. The estimator
consistently estimates the latent inverse Pearson correlation matrix uniformly
in both the index variable and kernel bandwidth. Its rate of convergence is
shown to be minimax optimal. Our method is supported by thorough numerical
simulations and an application to a neural imaging data set.
Regression-aware decompositions
Authors: Mark Tygert
Date: 2017-10-11 00:00:00
Tasks: Dimensionality Reduction, regression

Linear least-squares regression with a "design" matrix A approximates a given
matrix B via minimization of the spectral- or Frobenius-norm discrepancy
||AX-B|| over every conformingly sized matrix X. Another popular approximation
is low-rank approximation via principal component analysis (PCA) -- which is
essentially singular value decomposition (SVD) -- or interpolative
decomposition (ID). Classically, PCA/SVD and ID operate solely with the matrix
B being approximated, not supervised by any auxiliary matrix A. However, linear
least-squares regression models can inform the ID, yielding regression-aware
ID. As a bonus, this provides an interpretation as regression-aware PCA for a
kind of canonical correlation analysis between A and B. The regression-aware
decompositions effectively enable supervision to inform classical
dimensionality reduction, which classically has been totally unsupervised. The
regression-aware decompositions reveal the structure inherent in B that is
relevant to regression against A.
Few-Shot Learning with Metric-Agnostic Conditional Embeddings
Authors: Nathan Hilliard, Lawrence Phillips, Scott Howland, Artëm Yankov, Courtney D. Corley, Nathan O. Hodas
Date: 2018-02-12 00:00:00
Tasks: Few-Shot Learning, General Classification, Metric Learning

Learning high quality class representations from few examples is a key
problem in metric-learning approaches to few-shot learning. To accomplish this,
we introduce a novel architecture where class representations are conditioned
for each few-shot trial based on a target image. We also deviate from
traditional metric-learning approaches by training a network to perform
comparisons between classes rather than relying on a static metric comparison.
This allows the network to decide what aspects of each class are important for
the comparison at hand. We find that this flexible architecture works well in
practice, achieving state-of-the-art performance on the Caltech-UCSD birds
fine-grained classification task.
Artistic style transfer for videos and spherical images
Authors: Manuel Ruder, Alexey Dosovitskiy, Thomas Brox
Date: 2017-08-13 00:00:00
Tasks: Style Transfer

Manually re-drawing an image in a certain artistic style takes a professional
artist a long time. Doing this for a video sequence single-handedly is beyond
imagination. We present two computational approaches that transfer the style
from one image (for example, a painting) to a whole video sequence. In our
first approach, we adapt to videos the original image style transfer technique
by Gatys et al. based on energy minimization. We introduce new ways of
initialization and new loss functions to generate consistent and stable
stylized video sequences even in cases with large motion and strong occlusion.
Our second approach formulates video stylization as a learning problem. We
propose a deep network architecture and training procedures that allow us to
stylize arbitrary-length videos in a consistent and stable way, and nearly in
real time. We show that the proposed methods clearly outperform simpler
baselines both qualitatively and quantitatively. Finally, we propose a way to
adapt these approaches also to 360 degree images and videos as they emerge with
recent virtual reality hardware.
Learning a Neural-network-based Representation for Open Set Recognition
Authors: Mehadi Hassen, Philip K. Chan
Date: 2018-02-12 00:00:00
Tasks: General Classification, Malware Classification, Open Set Learning

Open set recognition problems exist in many domains. For example in security,
new malware classes emerge regularly; therefore malware classification systems
need to identify instances from unknown classes in addition to discriminating
between known classes. In this paper we present a neural network based
representation for addressing the open set recognition problem. In this
representation instances from the same class are close to each other while
instances from different classes are further apart, resulting in statistically
significant improvement when compared to other approaches on three datasets
from two different domains.
A Unified Implicit Dialog Framework for Conversational Search
Authors: Song Feng, R. Chulaka Gunasekara, Sunil Shashidhara, Kshitij P. Fadnis, Lazaros C. Polymenakos
Date: 2018-02-12 00:00:00
Tasks: Conversational Search

We propose a unified Implicit Dialog framework for goal-oriented, information
seeking tasks of Conversational Search applications. It aims to enable dialog
interactions with domain data without replying on explicitly encoded the rules
but utilizing the underlying data representation to build the components
required for dialog interaction, which we refer as Implicit Dialog in this
work. The proposed framework consists of a pipeline of End-to-End trainable
modules. A centralized knowledge representation is used to semantically ground
multiple dialog modules. An associated set of tools are integrated with the
framework to gather end users' input for continuous improvement of the system.
The goal is to facilitate development of conversational systems by identifying
the components and the data that can be adapted and reused across many end-user
applications. We demonstrate our approach by creating conversational agents for
several independent domains.
Kernel Meets Sieve: Post-Regularization Confidence Bands for Sparse Additive Model
Authors: Junwei Lu, Mladen Kolar, Han Liu
Date: 2015-03-10 00:00:00
Tasks: Additive models

We develop a novel procedure for constructing confidence bands for components
of a sparse additive model. Our procedure is based on a new kernel-sieve hybrid
estimator that combines two most popular nonparametric estimation methods in
the literature, the kernel regression and the spline method, and is of interest
in its own right. Existing methods for fitting sparse additive model are
primarily based on sieve estimators, while the literature on confidence bands
for nonparametric models are primarily based upon kernel or local polynomial
estimators. Our kernel-sieve hybrid estimator combines the best of both worlds
and allows us to provide a simple procedure for constructing confidence bands
in high-dimensional sparse additive models. We prove that the confidence bands
are asymptotically honest by studying approximation with a Gaussian process.
Thorough numerical results on both synthetic data and real-world neuroscience
data are provided to demonstrate the efficacy of the theory.
Gaining Free or Low-Cost Transparency with Interpretable Partial Substitute
Authors: Tong Wang
Date: 2018-02-12 00:00:00
Tasks: Decision Making, Interpretable Machine Learning

This work addresses the situation where a black-box model with good predictive performance is chosen over its interpretable competitors, and we show interpretability is still achievable in this case. Our solution is to find an interpretable substitute on a subset of data where the black-box model is overkill or nearly overkill while leaving the rest to the black-box. This transparency is obtained at minimal cost or no cost of the predictive performance. Under this framework, we develop a Hybrid Rule Sets (HyRS) model that uses decision rules to capture the subspace of data where the rules are as accurate or almost as accurate as the black-box provided. To train a HyRS, we devise an efficient search algorithm that iteratively finds the optimal model and exploits theoretically grounded strategies to reduce computation. Our framework is agnostic to the black-box during training. Experiments on structured and text data show that HyRS obtains an effective trade-off between transparency and interpretability.
Multi-Armed Bandits on Partially Revealed Unit Interval Graphs
Authors: Xiao Xu, Sattar Vakili, Qing Zhao, Ananthram Swami
Date: 2018-02-12 00:00:00
Tasks: Multi-Armed Bandits

A stochastic multi-armed bandit problem with side information on the similarity and dissimilarity across different arms is considered. The action space of the problem can be represented by a unit interval graph (UIG) where each node represents an arm and the presence (absence) of an edge between two nodes indicates similarity (dissimilarity) between their mean rewards. Two settings of complete and partial side information based on whether the UIG is fully revealed are studied and a general two-step learning structure consisting of an offline reduction of the action space and online aggregation of reward observations from similar arms is proposed to fully exploit the topological structure of the side information. In both cases, the computation efficiency and the order optimality of the proposed learning policies in terms of both the size of the action space and the time length are established.
Neural Program Search: Solving Programming Tasks from Description and Examples
Authors: Illia Polosukhin, Alexander Skidanov
Date: 2018-02-12 00:00:00
Tasks: Program Synthesis

We present a Neural Program Search, an algorithm to generate programs from
natural language description and a small number of input/output examples. The
algorithm combines methods from Deep Learning and Program Synthesis fields by
designing rich domain-specific language (DSL) and defining efficient search
algorithm guided by a Seq2Tree model on it. To evaluate the quality of the
approach we also present a semi-synthetic dataset of descriptions with test
examples and corresponding programs. We show that our algorithm significantly
outperforms a sequence-to-sequence model with attention baseline.
Stochastic quasi-Newton with adaptive step lengths for large-scale problems
Authors: Adrian Wills, Thomas Schön
Date: 2018-02-12 00:00:00

We provide a numerically robust and fast method capable of exploiting the
local geometry when solving large-scale stochastic optimisation problems. Our
key innovation is an auxiliary variable construction coupled with an inverse
Hessian approximation computed using a receding history of iterates and
gradients. It is the Markov chain nature of the classic stochastic gradient
algorithm that enables this development. The construction offers a mechanism
for stochastic line search adapting the step length. We numerically evaluate
and compare against current state-of-the-art with encouraging performance on
real-world benchmark problems where the number of observations and unknowns is
in the order of millions.
Sparse Reject Option Classifier Using Successive Linear Programming
Authors: Kulin Shah, Naresh Manwani
Date: 2018-02-12 00:00:00

In this paper, we propose an approach for learning sparse reject option
classifiers using double ramp loss $L_{dr}$. We use DC programming to find the
risk minimizer. The algorithm solves a sequence of linear programs to learn the
reject option classifier. We show that the loss $L_{dr}$ is Fisher consistent.
We also show that the excess risk of loss $L_d$ is upper bounded by the excess
risk of $L_{dr}$. We derive the generalization error bounds for the proposed
approach. We show the effectiveness of the proposed approach by experimenting
it on several real world datasets. The proposed approach not only performs
comparable to the state of the art but it also successfully learns sparse
classifiers.
Generalization of Deep Neural Networks for Chest Pathology Classification in X-Rays Using Generative Adversarial Networks
Authors: Hojjat Salehinejad, Shahrokh Valaee, Tim Dowdell, Errol Colak, Joseph Barfett
Date: 2017-11-08 00:00:00
Tasks: General Classification, Generative Adversarial Network

Medical datasets are often highly imbalanced with over-representation of
common medical problems and a paucity of data from rare conditions. We propose
simulation of pathology in images to overcome the above limitations. Using
chest X-rays as a model medical image, we implement a generative adversarial
network (GAN) to create artificial images based upon a modest sized labeled
dataset. We employ a combination of real and artificial images to train a deep
convolutional neural network (DCNN) to detect pathology across five classes of
chest X-rays. Furthermore, we demonstrate that augmenting the original
imbalanced dataset with GAN generated images improves performance of chest
pathology classification using the proposed DCNN in comparison to the same DCNN
trained with the original dataset alone. This improved performance is largely
attributed to balancing of the dataset using GAN generated images, where image
classes that are lacking in example images are preferentially augmented.
SparseMAP: Differentiable Sparse Structured Inference
Authors: Vlad Niculae, André F. T. Martins, Mathieu Blondel, Claire Cardie
Date: 2018-02-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Dependency Parsing, Natural Language Inference, Structured Prediction

Structured prediction requires searching over a combinatorial number of
structures. To tackle it, we introduce SparseMAP: a new method for sparse
structured inference, and its natural loss function. SparseMAP automatically
selects only a few global structures: it is situated between MAP inference,
which picks a single structure, and marginal inference, which assigns
probability mass to all structures, including implausible ones. Importantly,
SparseMAP can be computed using only calls to a MAP oracle, making it
applicable to problems with intractable marginal inference, e.g., linear
assignment. Sparsity makes gradient backpropagation efficient regardless of the
structure, enabling us to augment deep neural networks with generic and sparse
structured hidden layers. Experiments in dependency parsing and natural
language inference reveal competitive accuracy, improved interpretability, and
the ability to capture natural language ambiguities, which is attractive for
pipeline systems.
Image-based Synthesis for Deep 3D Human Pose Estimation
Authors: Grégory Rogez, Cordelia Schmid
Date: 2018-02-12 00:00:00
Tasks: 3D Human Pose Estimation, 3D Pose Estimation, Domain Adaptation, Pose Estimation

This paper addresses the problem of 3D human pose estimation in the wild. A
significant challenge is the lack of training data, i.e., 2D images of humans
annotated with 3D poses. Such data is necessary to train state-of-the-art CNN
architectures. Here, we propose a solution to generate a large set of
photorealistic synthetic images of humans with 3D pose annotations. We
introduce an image-based synthesis engine that artificially augments a dataset
of real images with 2D human pose annotations using 3D motion capture data.
Given a candidate 3D pose, our algorithm selects for each joint an image whose
2D pose locally matches the projected 3D pose. The selected images are then
combined to generate a new synthetic image by stitching local image patches in
a kinematically constrained manner. The resulting images are used to train an
end-to-end CNN for full-body 3D pose estimation. We cluster the training data
into a large number of pose classes and tackle pose estimation as a $K$-way
classification problem. Such an approach is viable only with large training
sets such as ours. Our method outperforms most of the published works in terms
of 3D pose estimation in controlled environments (Human3.6M) and shows
promising results for real-world images (LSP). This demonstrates that CNNs
trained on artificial images generalize well to real images. Compared to data
generated from more classical rendering engines, our synthetic images do not
require any domain adaptation or fine-tuning stage.
Adaptive Rate of Convergence of Thompson Sampling for Gaussian Process Optimization
Authors: Kinjal Basu, Souvik Ghosh
Date: 2017-05-18 00:00:00
Tasks: global-optimization, Thompson Sampling

We consider the problem of global optimization of a function over a continuous domain. In our setup, we can evaluate the function sequentially at points of our choice and the evaluations are noisy. We frame it as a continuum-armed bandit problem with a Gaussian Process prior on the function. In this regime, most algorithms have been developed to minimize some form of regret. In this paper, we study the convergence of the sequential point $x^t$ to the global optimizer $x^*$ for the Thompson Sampling approach. Under some assumptions and regularity conditions, we prove concentration bounds for $x^t$ where the probability that $x^t$ is bounded away from $x^*$ decays exponentially fast in $t$. Moreover, the result allows us to derive adaptive convergence rates depending on the function structure.
Adversarial Audio Synthesis
Authors: Chris Donahue, Julian McAuley, Miller Puckette
Date: 2018-02-12 00:00:00
Proceeding: ICLR 2019 5
Tasks: Audio Generation, Audio Synthesis, Image Generation

Audio signals are sampled at high temporal resolutions, and learning to
synthesize audio requires capturing structure across a range of timescales.
Generative adversarial networks (GANs) have seen wide success at generating
images that are both locally and globally coherent, but they have seen little
application to audio generation. In this paper we introduce WaveGAN, a first
attempt at applying GANs to unsupervised synthesis of raw-waveform audio.
WaveGAN is capable of synthesizing one second slices of audio waveforms with
global coherence, suitable for sound effect generation. Our experiments
demonstrate that, without labels, WaveGAN learns to produce intelligible words
when trained on a small-vocabulary speech dataset, and can also synthesize
audio from other domains such as drums, bird vocalizations, and piano. We
compare WaveGAN to a method which applies GANs designed for image generation on
image-like audio feature representations, finding both approaches to be
promising.
Fast Interactive Image Retrieval using large-scale unlabeled data
Authors: Akshay Mehra, Jihun Hamm, Mikhail Belkin
Date: 2018-02-12 00:00:00
Tasks: Active Learning, Binary Classification, Image Retrieval, Retrieval

An interactive image retrieval system learns which images in the database
belong to a user's query concept, by analyzing the example images and feedback
provided by the user. The challenge is to retrieve the relevant images with
minimal user interaction. In this work, we propose to solve this problem by
posing it as a binary classification task of classifying all images in the
database as being relevant or irrelevant to the user's query concept. Our
method combines active learning with graph-based semi-supervised learning
(GSSL) to tackle this problem. Active learning reduces the number of user
interactions by querying the labels of the most informative points and GSSL
allows to use abundant unlabeled data along with the limited labeled data
provided by the user. To efficiently find the most informative point, we use an
uncertainty sampling based method that queries the label of the point nearest
to the decision boundary of the classifier. We estimate this decision boundary
using our heuristic of adaptive threshold. To utilize huge volumes of unlabeled
data we use an efficient approximation based method that reduces the complexity
of GSSL from $O(n^3)$ to $O(n)$, making GSSL scalable. We make the classifier
robust to the diversity and noisy labels associated with images in large
databases by incorporating information from multiple modalities such as visual
information extracted from deep learning based models and semantic information
extracted from the WordNet. High F1 scores within few relevance feedback rounds
in our experiments with concepts defined on AnimalWithAttributes and Imagenet
(1.2 million images) datasets indicate the effectiveness and scalability of our
approach.
End-to-End Automatic Speech Translation of Audiobooks
Authors: Alexandre Bérard, Laurent Besacier, Ali Can Kocabiyikoglu, Olivier Pietquin
Date: 2018-02-12 00:00:00
Tasks: automatic-speech-translation, Speech-to-Text, Speech-to-Text Translation, Translation

We investigate end-to-end speech-to-text translation on a corpus of
audiobooks specifically augmented for this task. Previous works investigated
the extreme case where source language transcription is not available during
learning nor decoding, but we also study a midway case where source language
transcription is available at training time only. In this case, a single model
is trained to decode source speech into target text in a single pass.
Experimental results show that it is possible to train compact and efficient
end-to-end speech translation models in this setup. We also distribute the
corpus and hope that our speech translation baseline on this corpus will be
challenged in the future.
client2vec: Towards Systematic Baselines for Banking Applications
Authors: Leonardo Baldassini, Jose Antonio Rodríguez Serrano
Date: 2018-02-12 00:00:00
Tasks: Denoising, Feature Engineering, Model Selection

The workflow of data scientists normally involves potentially inefficient
processes such as data mining, feature engineering and model selection. Recent
research has focused on automating this workflow, partly or in its entirety, to
improve productivity. We choose the former approach and in this paper share our
experience in designing the client2vec: an internal library to rapidly build
baselines for banking applications. Client2vec uses marginalized stacked
denoising autoencoders on current account transactions data to create vector
embeddings which represent the behaviors of our clients. These representations
can then be used in, and optimized against, a variety of tasks such as client
segmentation, profiling and targeting. Here we detail how we selected the
algorithmic machinery of client2vec and the data it works on and present
experimental results on several business cases.
Electric Vehicle Driver Clustering using Statistical Model and Machine Learning
Authors: Yingqi Xiong, Bin Wang, Chi-Cheng Chu, Rajit Gadh
Date: 2018-02-12 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, energy management, Load Forecasting, Management, Scheduling

Electric Vehicle (EV) is playing a significant role in the distribution
energy management systems since the power consumption level of the EVs is much
higher than the other regular home appliances. The randomness of the EV driver
behaviors make the optimal charging or discharging scheduling even more
difficult due to the uncertain charging session parameters. To minimize the
impact of behavioral uncertainties, it is critical to develop effective methods
to predict EV load for smart EV energy management. Using the EV smart charging
infrastructures on UCLA campus and city of Santa Monica as testbeds, we have
collected real-world datasets of EV charging behaviors, based on which we
proposed an EV user modeling technique which combines statistical analysis and
machine learning approaches. Specifically, unsupervised clustering algorithm,
and multilayer perceptron are applied to historical charging record to make the
day-ahead EV parking and load prediction. Experimental results with
cross-validation show that our model can achieve good performance for charging
control scheduling and online EV load forecasting.