pwc-archive/papers-with-abstracts huggingface dataset:
Lexis: An Optimization Framework for Discovering the Hierarchical Structure of Sequential Data
Authors: Payam Siyari, Bistra Dilkina, Constantine Dovrolis
Date: 2016-02-17 00:00:00
Tasks: Text Compression

Data represented as strings abounds in biology, linguistics, document mining,
web search and many other fields. Such data often have a hierarchical
structure, either because they were artificially designed and composed in a
hierarchical manner or because there is an underlying evolutionary process that
creates repeatedly more complex strings from simpler substrings. We propose a
framework, referred to as "Lexis", that produces an optimized hierarchical
representation of a given set of "target" strings. The resulting hierarchy,
"Lexis-DAG", shows how to construct each target through the concatenation of
intermediate substrings, minimizing the total number of such concatenations or
DAG edges. The Lexis optimization problem is related to the smallest grammar
problem. After we prove its NP-Hardness for two cost formulations, we propose
an efficient greedy algorithm for the construction of Lexis-DAGs. We also
consider the problem of identifying the set of intermediate nodes (substrings)
that collectively form the "core" of a Lexis-DAG, which is important in the
analysis of Lexis-DAGs. We show that the Lexis framework can be applied in
diverse applications such as optimized synthesis of DNA fragments in genomic
libraries, hierarchical structure discovery in protein sequences,
dictionary-based text compression, and feature extraction from a set of
documents.
Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder
Authors: Yu-An Chung, Chao-Chung Wu, Chia-Hao Shen, Hung-Yi Lee, Lin-shan Lee
Date: 2016-03-03 00:00:00
Tasks: Decoder, Denoising, Dynamic Time Warping

The vector representations of fixed dimensionality for words (in text)
offered by Word2Vec have been shown to be very useful in many application
scenarios, in particular due to the semantic information they carry. This paper
proposes a parallel version, the Audio Word2Vec. It offers the vector
representations of fixed dimensionality for variable-length audio segments.
These vector representations are shown to describe the sequential phonetic
structures of the audio segments to a good degree, with very attractive real
world applications such as query-by-example Spoken Term Detection (STD). In
this STD application, the proposed approach significantly outperformed the
conventional Dynamic Time Warping (DTW) based approaches at significantly lower
computation requirements. We propose unsupervised learning of Audio Word2Vec
from audio data without human annotation using Sequence-to-sequence Audoencoder
(SA). SA consists of two RNNs equipped with Long Short-Term Memory (LSTM)
units: the first RNN (encoder) maps the input audio sequence into a vector
representation of fixed dimensionality, and the second RNN (decoder) maps the
representation back to the input audio sequence. The two RNNs are jointly
trained by minimizing the reconstruction error. Denoising Sequence-to-sequence
Autoencoder (DSA) is furthered proposed offering more robust learning.
Discovering Neuronal Cell Types and Their Gene Expression Profiles Using a Spatial Point Process Mixture Model
Authors: Furong Huang, Animashree Anandkumar, Christian Borgs, Jennifer Chayes, Ernest Fraenkel, Michael Hawrylycz, Ed Lein, Alessandro Ingrosso, Srinivas Turaga
Date: 2016-02-04 00:00:00

Cataloging the neuronal cell types that comprise circuitry of individual
brain regions is a major goal of modern neuroscience and the BRAIN initiative.
Single-cell RNA sequencing can now be used to measure the gene expression
profiles of individual neurons and to categorize neurons based on their gene
expression profiles. While the single-cell techniques are extremely powerful
and hold great promise, they are currently still labor intensive, have a high
cost per cell, and, most importantly, do not provide information on spatial
distribution of cell types in specific regions of the brain. We propose a
complementary approach that uses computational methods to infer the cell types
and their gene expression profiles through analysis of brain-wide single-cell
resolution in situ hybridization (ISH) imagery contained in the Allen Brain
Atlas (ABA). We measure the spatial distribution of neurons labeled in the ISH
image for each gene and model it as a spatial point process mixture, whose
mixture weights are given by the cell types which express that gene. By fitting
a point process mixture model jointly to the ISH images, we infer both the
spatial point process distribution for each cell type and their gene expression
profile. We validate our predictions of cell type-specific gene expression
profiles using single cell RNA sequencing data, recently published for the
mouse somatosensory cortex. Jointly with the gene expression profiles, cell
features such as cell size, orientation, intensity and local density level are
inferred per cell type.
Dynamic Frame skip Deep Q Network
Authors: Aravind Srinivas, Sahil Sharma, Balaraman Ravindran
Date: 2016-05-17 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, Reinforcement Learning

Deep Reinforcement Learning methods have achieved state of the art performance in learning control policies for the games in the Atari 2600 domain. One of the important parameters in the Arcade Learning Environment (ALE) is the frame skip rate. It decides the granularity at which agents can control game play. A frame skip value of $k$ allows the agent to repeat a selected action $k$ number of times. The current state of the art architectures like Deep Q-Network (DQN) and Dueling Network Architectures (DuDQN) consist of a framework with a static frame skip rate, where the action output from the network is repeated for a fixed number of frames regardless of the current state. In this paper, we propose a new architecture, Dynamic Frame skip Deep Q-Network (DFDQN) which makes the frame skip rate a dynamic learnable parameter. This allows us to choose the number of times an action is to be repeated based on the current state. We show empirically that such a setting improves the performance on relatively harder games like Seaquest.
Distributed Machine Learning in Materials that Couple Sensing, Actuation, Computation and Communication
Authors: Dana Hughes, Nikolaus Correll
Date: 2016-06-11 00:00:00
Tasks: BIG-bench Machine Learning, Structural Health Monitoring, Survey

This paper reviews machine learning applications and approaches to detection,
classification and control of intelligent materials and structures with
embedded distributed computation elements. The purpose of this survey is to
identify desired tasks to be performed in each type of material or structure
(e.g., damage detection in composites), identify and compare common approaches
to learning such tasks, and investigate models and training paradigms used.
Machine learning approaches and common temporal features used in the domains of
structural health monitoring, morphable aircraft, wearable computing and
robotic skins are explored. As the ultimate goal of this research is to
incorporate the approaches described in this survey into a robotic material
paradigm, the potential for adapting the computational models used in these
applications, and corresponding training algorithms, to an amorphous network of
computing nodes is considered. Distributed versions of support vector machines,
graphical models and mixture models developed in the field of wireless sensor
networks are reviewed. Potential areas of investigation, including possible
architectures for incorporating machine learning into robotic nodes, training
approaches, and the possibility of using deep learning approaches for automatic
feature extraction, are discussed.
Incoherent Tensor Norms and Their Applications in Higher Order Tensor Completion
Authors: Ming Yuan, Cun-Hui Zhang
Date: 2016-06-10 00:00:00

In this paper, we investigate the sample size requirement for a general class
of nuclear norm minimization methods for higher order tensor completion. We
introduce a class of tensor norms by allowing for different levels of
coherence, which allows us to leverage the incoherence of a tensor. In
particular, we show that a $k$th order tensor of rank $r$ and dimension
$d\times\cdots\times d$ can be recovered perfectly from as few as
$O((r^{(k-1)/2}d^{3/2}+r^{k-1}d)(\log(d))^2)$ uniformly sampled entries through
an appropriate incoherent nuclear norm minimization. Our results demonstrate
some key differences between completing a matrix and a higher order tensor:
They not only point to potential room for improvement over the usual nuclear
norm minimization but also highlight the importance of explicitly accounting
for incoherence, when dealing with higher order tensors.
XGBoost: A Scalable Tree Boosting System
Authors: Tianqi Chen, Carlos Guestrin
Date: 2016-03-09 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Data Compression, Dimensionality Reduction, General Classification, Humor Detection, Video Super-Resolution

Tree boosting is a highly effective and widely used machine learning method.
In this paper, we describe a scalable end-to-end tree boosting system called
XGBoost, which is used widely by data scientists to achieve state-of-the-art
results on many machine learning challenges. We propose a novel sparsity-aware
algorithm for sparse data and weighted quantile sketch for approximate tree
learning. More importantly, we provide insights on cache access patterns, data
compression and sharding to build a scalable tree boosting system. By combining
these insights, XGBoost scales beyond billions of examples using far fewer
resources than existing systems.
Improved Techniques for Training GANs
Authors: Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen
Date: 2016-06-10 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Conditional Image Generation, Image Generation, Semi-Supervised Image Classification

We present a variety of new architectural features and training procedures
that we apply to the generative adversarial networks (GANs) framework. We focus
on two applications of GANs: semi-supervised learning, and the generation of
images that humans find visually realistic. Unlike most work on generative
models, our primary goal is not to train a model that assigns high likelihood
to test data, nor do we require the model to be able to learn well without
using any labels. Using our new techniques, we achieve state-of-the-art results
in semi-supervised classification on MNIST, CIFAR-10 and SVHN. The generated
images are of high quality as confirmed by a visual Turing test: our model
generates MNIST samples that humans cannot distinguish from real data, and
CIFAR-10 samples that yield a human error rate of 21.3%. We also present
ImageNet samples with unprecedented resolution and show that our methods enable
the model to learn recognizable features of ImageNet classes.
Position-Indexed Formulations for Kidney Exchange
Authors: John P. Dickerson, David F. Manlove, Benjamin Plaut, Tuomas Sandholm, James Trimble
Date: 2016-06-06 00:00:00
Tasks: Position

A kidney exchange is an organized barter market where patients in need of a
kidney swap willing but incompatible donors. Determining an optimal set of
exchanges is theoretically and empirically hard. Traditionally, exchanges took
place in cycles, with each participating patient-donor pair both giving and
receiving a kidney. The recent introduction of chains, where a donor without a
paired patient triggers a sequence of donations without requiring a kidney in
return, increased the efficacy of fielded kidney exchanges---while also
dramatically raising the empirical computational hardness of clearing the
market in practice. While chains can be quite long, unbounded-length chains are
not desirable: planned donations can fail before transplant for a variety of
reasons, and the failure of a single donation causes the rest of that chain to
fail, so parallel shorter chains are better in practice.
  In this paper, we address the tractable clearing of kidney exchanges with
short cycles and chains that are long but bounded. This corresponds to the
practice at most modern fielded kidney exchanges. We introduce three new
integer programming formulations, two of which are compact. Furthermore, one of
these models has a linear programming relaxation that is exactly as tight as
the previous tightest formulation (which was not compact) for instances in
which each donor has a paired patient. On real data from the UNOS nationwide
exchange in the United States and the NLDKSS nationwide exchange in the United
Kingdom, as well as on generated realistic large-scale data, we show that our
new models are competitive with all existing solvers---in many cases
outperforming all other solvers by orders of magnitude.
A Diversity-Promoting Objective Function for Neural Conversation Models
Authors: Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, Bill Dolan
Date: 2015-10-11 00:00:00
Proceeding: NAACL 2016 6
Tasks: Conversational Response Generation, Diversity, Response Generation

Sequence-to-sequence neural network models for generation of conversational
responses tend to generate safe, commonplace responses (e.g., "I don't know")
regardless of the input. We suggest that the traditional objective function,
i.e., the likelihood of output (response) given input (message) is unsuited to
response generation tasks. Instead we propose using Maximum Mutual Information
(MMI) as the objective function in neural models. Experimental results
demonstrate that the proposed MMI models produce more diverse, interesting, and
appropriate responses, yielding substantive gains in BLEU scores on two
conversational datasets and in human evaluations.
Kalman-based Stochastic Gradient Method with Stop Condition and Insensitivity to Conditioning
Authors: Vivak Patel
Date: 2015-12-03 00:00:00
Tasks: regression

Modern proximal and stochastic gradient descent (SGD) methods are believed to
efficiently minimize large composite objective functions, but such methods have
two algorithmic challenges: (1) a lack of fast or justified stop conditions,
and (2) sensitivity to the objective function's conditioning. In response to
the first challenge, modern proximal and SGD methods guarantee convergence only
after multiple epochs, but such a guarantee renders proximal and SGD methods
infeasible when the number of component functions is very large or infinite. In
response to the second challenge, second order SGD methods have been developed,
but they are marred by the complexity of their analysis. In this work, we
address these challenges on the limited, but important, linear regression
problem by introducing and analyzing a second order proximal/SGD method based
on Kalman Filtering (kSGD). Through our analysis, we show kSGD is
asymptotically optimal, develop a fast algorithm for very large, infinite or
streaming data sources with a justified stop condition, prove that kSGD is
insensitive to the problem's conditioning, and develop a unique approach for
analyzing the complex second order dynamics. Our theoretical results are
supported by numerical experiments on three regression problems (linear,
nonparametric wavelet, and logistic) using three large publicly available
datasets. Moreover, our analysis and experiments lay a foundation for embedding
kSGD in multiple epoch algorithms, extending kSGD to other problem classes, and
developing parallel and low memory kSGD implementations.
Convolutional Radio Modulation Recognition Networks
Authors: Timothy J. O'Shea, Johnathan Corgan, T. Charles Clancy
Date: 2016-02-12 00:00:00
Tasks: General Classification, Time Series, Time Series Analysis

We study the adaptation of convolutional neural networks to the complex
temporal radio signal domain. We compare the efficacy of radio modulation
classification using naively learned features against using expert features
which are widely used in the field today and we show significant performance
improvements. We show that blind temporal learning on large and densely encoded
time series using deep convolutional neural networks is viable and a strong
candidate approach for this task especially at low signal to noise ratio.
Multi-object Classification via Crowdsourcing with a Reject Option
Authors: Qunwei Li, Aditya Vempaty, Lav R. Varshney, Pramod K. Varshney
Date: 2016-02-01 00:00:00
Tasks: Classification, General Classification, Object

Consider designing an effective crowdsourcing system for an $M$-ary
classification task. Crowd workers complete simple binary microtasks whose
results are aggregated to give the final result. We consider the novel scenario
where workers have a reject option so they may skip microtasks when they are
unable or choose not to respond. For example, in mismatched speech
transcription, workers who do not know the language may not be able to respond
to microtasks focused on phonological dimensions outside their categorical
perception. We present an aggregation approach using a weighted majority voting
rule, where each worker's response is assigned an optimized weight to maximize
the crowd's classification performance. We evaluate system performance in both
exact and asymptotic forms. Further, we consider the setting where there may be
a set of greedy workers that complete microtasks even when they are unable to
perform it reliably. We consider an oblivious and an expurgation strategy to
deal with greedy workers, developing an algorithm to adaptively switch between
the two based on the estimated fraction of greedy workers in the anonymous
crowd. Simulation results show improved performance compared with conventional
majority voting.
Generative Adversarial Imitation Learning
Authors: Jonathan Ho, Stefano Ermon
Date: 2016-06-10 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Imitation Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Consider learning a policy from example expert behavior, without interaction
with the expert or access to reinforcement signal. One approach is to recover
the expert's cost function with inverse reinforcement learning, then extract a
policy from that cost function with reinforcement learning. This approach is
indirect and can be slow. We propose a new general framework for directly
extracting a policy from data, as if it were obtained by reinforcement learning
following inverse reinforcement learning. We show that a certain instantiation
of our framework draws an analogy between imitation learning and generative
adversarial networks, from which we derive a model-free imitation learning
algorithm that obtains significant performance gains over existing model-free
methods in imitating complex behaviors in large, high-dimensional environments.
De-identification of Patient Notes with Recurrent Neural Networks
Authors: Franck Dernoncourt, Ji Young Lee, Ozlem Uzuner, Peter Szolovits
Date: 2016-06-10 00:00:00
Tasks: De-identification, Feature Engineering

Objective: Patient notes in electronic health records (EHRs) may contain
critical information for medical investigations. However, the vast majority of
medical investigators can only access de-identified notes, in order to protect
the confidentiality of patients. In the United States, the Health Insurance
Portability and Accountability Act (HIPAA) defines 18 types of protected health
information (PHI) that needs to be removed to de-identify patient notes. Manual
de-identification is impractical given the size of EHR databases, the limited
number of researchers with access to the non-de-identified notes, and the
frequent mistakes of human annotators. A reliable automated de-identification
system would consequently be of high value.
  Materials and Methods: We introduce the first de-identification system based
on artificial neural networks (ANNs), which requires no handcrafted features or
rules, unlike existing systems. We compare the performance of the system with
state-of-the-art systems on two datasets: the i2b2 2014 de-identification
challenge dataset, which is the largest publicly available de-identification
dataset, and the MIMIC de-identification dataset, which we assembled and is
twice as large as the i2b2 2014 dataset.
  Results: Our ANN model outperforms the state-of-the-art systems. It yields an
F1-score of 97.85 on the i2b2 2014 dataset, with a recall 97.38 and a precision
of 97.32, and an F1-score of 99.23 on the MIMIC de-identification dataset, with
a recall 99.25 and a precision of 99.06.
  Conclusion: Our findings support the use of ANNs for de-identification of
patient notes, as they show better performance than previously published
systems while requiring no feature engineering.
Face Detection with the Faster R-CNN
Authors: Huaizu Jiang, Erik Learned-Miller
Date: 2016-06-10 00:00:00
Tasks: Face Detection, object-detection, Object Detection

The Faster R-CNN has recently demonstrated impressive results on various
object detection benchmarks. By training a Faster R-CNN model on the large
scale WIDER face dataset, we report state-of-the-art results on two widely used
face detection benchmarks, FDDB and the recently released IJB-A.
Deep Directed Generative Models with Energy-Based Probability Estimation
Authors: Taesup Kim, Yoshua Bengio
Date: 2016-06-10 00:00:00

Training energy-based probabilistic models is confronted with apparently
intractable sums, whose Monte Carlo estimation requires sampling from the
estimated probability distribution in the inner loop of training. This can be
approximately achieved by Markov chain Monte Carlo methods, but may still face
a formidable obstacle that is the difficulty of mixing between modes with sharp
concentrations of probability. Whereas an MCMC process is usually derived from
a given energy function based on mathematical considerations and requires an
arbitrarily long time to obtain good and varied samples, we propose to train a
deep directed generative model (not a Markov chain) so that its sampling
distribution approximately matches the energy function that is being trained.
Inspired by generative adversarial networks, the proposed framework involves
training of two models that represent dual views of the estimated probability
distribution: the energy function (mapping an input configuration to a scalar
energy value) and the generator (mapping a noise vector to a generated
configuration), both represented by deep neural networks.
Scan Order in Gibbs Sampling: Models in Which it Matters and Bounds on How Much
Authors: Bryan He, Christopher De Sa, Ioannis Mitliagkas, Christopher Ré
Date: 2016-06-10 00:00:00
Proceeding: NeurIPS 2016 12

Gibbs sampling is a Markov Chain Monte Carlo sampling technique that
iteratively samples variables from their conditional distributions. There are
two common scan orders for the variables: random scan and systematic scan. Due
to the benefits of locality in hardware, systematic scan is commonly used, even
though most statistical guarantees are only for random scan. While it has been
conjectured that the mixing times of random scan and systematic scan do not
differ by more than a logarithmic factor, we show by counterexample that this
is not the case, and we prove that that the mixing times do not differ by more
than a polynomial factor under mild conditions. To prove these relative bounds,
we introduce a method of augmenting the state space to study systematic scan
using conductance.