pwc-archive/papers-with-abstracts huggingface dataset:
Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs
Authors: Stefan Hadjis, Ce Zhang, Ioannis Mitliagkas, Dan Iter, Christopher Ré
Date: 2016-06-14 00:00:00

We study the factors affecting training time in multi-device deep learning
systems. Given a specification of a convolutional neural network, our goal is
to minimize the time to train this model on a cluster of commodity CPUs and
GPUs. We first focus on the single-node setting and show that by using standard
batching and data-parallel techniques, throughput can be improved by at least
5.5x over state-of-the-art systems on CPUs. This ensures an end-to-end training
speed directly proportional to the throughput of a device regardless of its
underlying hardware, allowing each node in the cluster to be treated as a black
box. Our second contribution is a theoretical and empirical study of the
tradeoffs affecting end-to-end training time in a multiple-device setting. We
identify the degree of asynchronous parallelization as a key factor affecting
both hardware and statistical efficiency. We see that asynchrony can be viewed
as introducing a momentum term. Our results imply that tuning momentum is
critical in asynchronous parallel configurations, and suggest that published
results that have not been fully tuned might report suboptimal performance for
some configurations. For our third contribution, we use our novel understanding
of the interaction between system and optimization dynamics to provide an
efficient hyperparameter optimizer. Our optimizer involves a predictive model
for the total time to convergence and selects an allocation of resources to
minimize that time. We demonstrate that the most popular distributed deep
learning systems fall within our tradeoff space, but do not optimize within the
space. By doing this optimization, our prototype runs 1.9x to 12x faster than
the fastest state-of-the-art systems.
Bidirectional LSTM-CRF for Clinical Concept Extraction
Authors: Raghavendra Chalapathy, Ehsan Zare Borzeshi, Massimo Piccardi
Date: 2016-10-19 00:00:00
Proceeding: WS 2016 12
Tasks: Clinical Concept Extraction, Word Embeddings

Extraction of concepts present in patient clinical records is an essential
step in clinical research. The 2010 i2b2/VA Workshop on Natural Language
Processing Challenges for clinical records presented concept extraction (CE)
task, with aim to identify concepts (such as treatments, tests, problems) and
classify them into predefined categories. State-of-the-art CE approaches
heavily rely on hand crafted features and domain specific resources which are
hard to collect and tune. For this reason, this paper employs bidirectional
LSTM with CRF decoding initialized with general purpose off-the-shelf word
embeddings for CE. The experimental results achieved on 2010 i2b2/VA reference
standard corpora using bidirectional LSTM CRF ranks closely with top ranked
systems.
Mixed context networks for semantic segmentation
Authors: Haiming Sun, Di Xie, ShiLiang Pu
Date: 2016-10-19 00:00:00
Tasks: General Classification, Segmentation, Semantic Segmentation

Semantic segmentation is challenging as it requires both object-level
information and pixel-level accuracy. Recently, FCN-based systems gained great
improvement in this area. Unlike classification networks, combining features of
different layers plays an important role in these dense prediction models, as
these features contains information of different levels. A number of models
have been proposed to show how to use these features. However, what is the best
architecture to make use of features of different layers is still a question.
In this paper, we propose a module, called mixed context network, and show that
our presented system outperforms most existing semantic segmentation systems by
making use of this module.
Improving Covariance-Regularized Discriminant Analysis for EHR-based Predictive Analytics of Diseases
Authors: Sijia Yang, Haoyi Xiong, Kaibo Xu, Licheng Wang, Jiang Bian, Zeyi Sun
Date: 2016-10-18 00:00:00
Tasks: Dimensionality Reduction, General Classification

Linear Discriminant Analysis (LDA) is a well-known technique for feature extraction and dimension reduction. The performance of classical LDA, however, significantly degrades on the High Dimension Low Sample Size (HDLSS) data for the ill-posed inverse problem. Existing approaches for HDLSS data classification typically assume the data in question are with Gaussian distribution and deal the HDLSS classification problem with regularization. However, these assumptions are too strict to hold in many emerging real-life applications, such as enabling personalized predictive analysis using Electronic Health Records (EHRs) data collected from an extremely limited number of patients who have been diagnosed with or without the target disease for prediction. In this paper, we revised the problem of predictive analysis of disease using personal EHR data and LDA classifier. To fill the gap, in this paper, we first studied an analytical model that understands the accuracy of LDA for classifying data with arbitrary distribution. The model gives a theoretical upper bound of LDA error rate that is controlled by two factors: (1) the statistical convergence rate of (inverse) covariance matrix estimators and (2) the divergence of the training/testing datasets to fitted distributions. To this end, we could lower the error rate by balancing the two factors for better classification performance. Hereby, we further proposed a novel LDA classifier De-Sparse that leverages De-sparsified Graphical Lasso to improve the estimation of LDA, which outperforms state-of-the-art LDA approaches developed for HDLSS data. Such advances and effectiveness are further demonstrated by both theoretical analysis and extensive experiments on EHR datasets.
Truthful Mechanisms for Matching and Clustering in an Ordinal World
Authors: Elliot Anshelevich, Shreyas Sekar
Date: 2016-10-13 00:00:00
Tasks: Clustering

We study truthful mechanisms for matching and related problems in a partial
information setting, where the agents' true utilities are hidden, and the
algorithm only has access to ordinal preference information. Our model is
motivated by the fact that in many settings, agents cannot express the
numerical values of their utility for different outcomes, but are still able to
rank the outcomes in their order of preference. Specifically, we study problems
where the ground truth exists in the form of a weighted graph of agent
utilities, but the algorithm can only elicit the agents' private information in
the form of a preference ordering for each agent induced by the underlying
weights. Against this backdrop, we design truthful algorithms to approximate
the true optimum solution with respect to the hidden weights. Our techniques
yield universally truthful algorithms for a number of graph problems: a
1.76-approximation algorithm for Max-Weight Matching, 2-approximation algorithm
for Max k-matching, a 6-approximation algorithm for Densest k-subgraph, and a
2-approximation algorithm for Max Traveling Salesman as long as the hidden
weights constitute a metric. We also provide improved approximation algorithms
for such problems when the agents are not able to lie about their preferences.
Our results are the first non-trivial truthful approximation algorithms for
these problems, and indicate that in many situations, we can design robust
algorithms even when the agents may lie and only provide ordinal information
instead of precise utilities.
Robot Vision Architecture for Autonomous Clothes Manipulation
Authors: Li Sun, Gerardo Aragon-Camarasa, Simon Rogers, J. Paul Siebert
Date: 2016-10-18 00:00:00

This paper presents a novel robot vision architecture for perceiving generic
3D clothes configurations. Our architecture is hierarchically structured,
starting from low-level curvatures, across mid-level geometric shapes \&
topology descriptions; and finally approaching high-level semantic surface
structure descriptions. We demonstrate our robot vision architecture in a
customised dual-arm industrial robot with our self-designed, off-the-self
stereo vision system, carrying out autonomous grasping and dual-arm flattening.
It is worth noting that the proposed dual-arm flattening approach is unique
among the state-of-the-art robot autonomous system, which is the major
contribution of this paper. The experimental results show that the proposed
dual-arm flattening using stereo vision system remarkably outperforms the
single-arm flattening and widely-cited Kinect-based sensing system for
dexterous manipulation tasks. In addition, the proposed grasping approach
achieves satisfactory performance on grasping various kind of garments,
verifying the capability of proposed visual perception architecture to be
adapted to more than one clothing manipulation tasks.
Fast, Scalable Phrase-Based SMT Decoding
Authors: Hieu Hoang, Nikolay Bogoychev, Lane Schwartz, Marcin Junczys-Dowmunt
Date: 2016-10-13 00:00:00
Proceeding: AMTA 2016 10
Tasks: Decoder, Machine Translation, Translation

The utilization of statistical machine translation (SMT) has grown enormously
over the last decade, many using open-source software developed by the NLP
community. As commercial use has increased, there is need for software that is
optimized for commercial requirements, in particular, fast phrase-based
decoding and more efficient utilization of modern multicore servers.
  In this paper we re-examine the major components of phrase-based decoding and
decoder implementation with particular emphasis on speed and scalability on
multicore machines. The result is a drop-in replacement for the Moses decoder
which is up to fifteen times faster and scales monotonically with the number of
cores.
Statistical Learning Theory Approach for Data Classification with l-diversity
Authors: Koray Mancuhan, Chris Clifton
Date: 2016-10-18 00:00:00
Tasks: Anatomy, Classification, Diversity, General Classification, Learning Theory

Corporations are retaining ever-larger corpuses of personal data; the
frequency or breaches and corresponding privacy impact have been rising
accordingly. One way to mitigate this risk is through use of anonymized data,
limiting the exposure of individual data to only where it is absolutely needed.
This would seem particularly appropriate for data mining, where the goal is
generalizable knowledge rather than data on specific individuals. In practice,
corporate data miners often insist on original data, for fear that they might
"miss something" with anonymized or differentially private approaches. This
paper provides a theoretical justification for the use of anonymized data.
Specifically, we show that a support vector classifier trained on anatomized
data satisfying l-diversity should be expected to do as well as on the original
data. Anatomy preserves all data values, but introduces uncertainty in the
mapping between identifying and sensitive values, thus satisfying l-diversity.
The theoretical effectiveness of the proposed approach is validated using
several publicly available datasets, showing that we outperform the state of
the art for support vector classification using training data protected by
k-anonymity, and are comparable to learning on the original data.
Decision Tree Classification on Outsourced Data
Authors: Koray Mancuhan, Chris Clifton
Date: 2016-10-18 00:00:00
Tasks: Classification, General Classification

This paper proposes a client-server decision tree learning method for
outsourced private data. The privacy model is anatomization/fragmentation: the
server sees data values, but the link between sensitive and identifying
information is encrypted with a key known only to clients. Clients have limited
processing and storage capability. Both sensitive and identifying information
thus are stored on the server. The approach presented also retains most
processing at the server, and client-side processing is amortized over
predictions made by the clients. Experiments on various datasets show that the
method produces decision trees approaching the accuracy of a non-private
decision tree, while substantially reducing the client's computing resource
requirements.
A polynomial-time relaxation of the Gromov-Hausdorff distance
Authors: Soledad Villar, Afonso S. Bandeira, Andrew J. Blumberg, Rachel Ward
Date: 2016-10-17 00:00:00

The Gromov-Hausdorff distance provides a metric on the set of isometry
classes of compact metric spaces. Unfortunately, computing this metric directly
is believed to be computationally intractable. Motivated by applications in
shape matching and point-cloud comparison, we study a semidefinite programming
relaxation of the Gromov-Hausdorff metric. This relaxation can be computed in
polynomial time, and somewhat surprisingly is itself a pseudometric. We
describe the induced topology on the set of compact metric spaces. Finally, we
demonstrate the numerical performance of various algorithms for computing the
relaxed distance and apply these algorithms to several relevant data sets. In
particular we propose a greedy algorithm for finding the best correspondence
between finite metric spaces that can handle hundreds of points.
Deep Motif Dashboard: Visualizing and Understanding Genomic Sequences Using Deep Neural Networks
Authors: Jack Lanchantin, Ritambhara Singh, Beilun Wang, Yanjun Qi
Date: 2016-08-12 00:00:00
Tasks: General Classification

Deep neural network (DNN) models have recently obtained state-of-the-art
prediction accuracy for the transcription factor binding (TFBS) site
classification task. However, it remains unclear how these approaches identify
meaningful DNA sequence signals and give insights as to why TFs bind to certain
locations. In this paper, we propose a toolkit called the Deep Motif Dashboard
(DeMo Dashboard) which provides a suite of visualization strategies to extract
motifs, or sequence patterns from deep neural network models for TFBS
classification. We demonstrate how to visualize and understand three important
DNN models: convolutional, recurrent, and convolutional-recurrent networks. Our
first visualization method is finding a test sequence's saliency map which uses
first-order derivatives to describe the importance of each nucleotide in making
the final prediction. Second, considering recurrent models make predictions in
a temporal manner (from one end of a TFBS sequence to the other), we introduce
temporal output scores, indicating the prediction score of a model over time
for a sequential input. Lastly, a class-specific visualization strategy finds
the optimal input sequence for a given TFBS positive class via stochastic
gradient optimization. Our experimental results indicate that a
convolutional-recurrent architecture performs the best among the three
architectures. The visualization techniques indicate that CNN-RNN makes
predictions by modeling both motifs as well as dependencies among them.
Modeling the Dynamics of Online Learning Activity
Authors: Charalampos Mavroforakis, Isabel Valera, Manuel Gomez Rodriguez
Date: 2016-10-18 00:00:00
Tasks: Clustering

People are increasingly relying on the Web and social media to find solutions
to their problems in a wide range of domains. In this online setting, closely
related problems often lead to the same characteristic learning pattern, in
which people sharing these problems visit related pieces of information,
perform almost identical queries or, more generally, take a series of similar
actions. In this paper, we introduce a novel modeling framework for clustering
continuous-time grouped streaming data, the hierarchical Dirichlet Hawkes
process (HDHP), which allows us to automatically uncover a wide variety of
learning patterns from detailed traces of learning activity. Our model allows
for efficient inference, scaling to millions of actions taken by thousands of
users. Experiments on real data gathered from Stack Overflow reveal that our
framework can recover meaningful learning patterns in terms of both content and
temporal dynamics, as well as accurately track users' interests and goals over
time.
RedQueen: An Online Algorithm for Smart Broadcasting in Social Networks
Authors: Ali Zarezade, Utkarsh Upadhyay, Hamid Rabiee, Manuel Gomez Rodriguez
Date: 2016-10-18 00:00:00

Users in social networks whose posts stay at the top of their followers'{}
feeds the longest time are more likely to be noticed. Can we design an online
algorithm to help them decide when to post to stay at the top? In this paper,
we address this question as a novel optimal control problem for jump stochastic
differential equations. For a wide variety of feed dynamics, we show that the
optimal broadcasting intensity for any user is surprisingly simple -- it is
given by the position of her most recent post on each of her follower's feeds.
As a consequence, we are able to develop a simple and highly efficient online
algorithm, RedQueen, to sample the optimal times for the user to post.
Experiments on both synthetic and real data gathered from Twitter show that our
algorithm is able to consistently make a user's posts more visible over time,
is robust to volume changes on her followers' feeds, and significantly
outperforms the state of the art.
Modeling community structure and topics in dynamic text networks
Authors: Teague Henry, David Banks, Christine Chai, Derek Owens-Oas
Date: 2016-10-18 00:00:00

The last decade has seen great progress in both dynamic network modeling and
topic modeling. This paper draws upon both areas to create a Bayesian method
that allows topic discovery to inform the latent network model and the network
structure to facilitate topic identification. We apply this method to the 467
top political blogs of 2012. Our results find complex community structure
within this set of blogs, where community membership depends strongly upon the
set of topics in which the blogger is interested.
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition
Authors: Jonathan Krause, Benjamin Sapp, Andrew Howard, Howard Zhou, Alexander Toshev, Tom Duerig, James Philbin, Li Fei-Fei
Date: 2015-11-20 00:00:00
Tasks: Active Learning, Fine-Grained Image Classification

Current approaches for fine-grained recognition do the following: First,
recruit experts to annotate a dataset of images, optionally also collecting
more structured data in the form of part annotations and bounding boxes.
Second, train a model utilizing this data. Toward the goal of solving
fine-grained recognition, we introduce an alternative approach, leveraging
free, noisy data from the web and simple, generic methods of recognition. This
approach has benefits in both performance and scalability. We demonstrate its
efficacy on four fine-grained datasets, greatly exceeding existing state of the
art without the manual collection of even a single label, and furthermore show
first results at scaling to more than 10,000 fine-grained categories.
Quantitatively, we achieve top-1 accuracies of 92.3% on CUB-200-2011, 85.4% on
Birdsnap, 93.4% on FGVC-Aircraft, and 80.8% on Stanford Dogs without using
their annotated training sets. We compare our approach to an active learning
approach for expanding fine-grained datasets.
Deep Amortized Inference for Probabilistic Programs
Authors: Daniel Ritchie, Paul Horsfall, Noah D. Goodman
Date: 2016-10-18 00:00:00
Tasks: Probabilistic Programming

Probabilistic programming languages (PPLs) are a powerful modeling tool, able
to represent any computable probability distribution. Unfortunately,
probabilistic program inference is often intractable, and existing PPLs mostly
rely on expensive, approximate sampling-based methods. To alleviate this
problem, one could try to learn from past inferences, so that future inferences
run faster. This strategy is known as amortized inference; it has recently been
applied to Bayesian networks and deep generative models. This paper proposes a
system for amortized inference in PPLs. In our system, amortization comes in
the form of a parameterized guide program. Guide programs have similar
structure to the original program, but can have richer data flow, including
neural network components. These networks can be optimized so that the guide
approximately samples from the posterior distribution defined by the original
program. We present a flexible interface for defining guide programs and a
stochastic gradient-based scheme for optimizing guide parameters, as well as
some preliminary results on automatically deriving guide programs. We explore
in detail the common machine learning pattern in which a 'local' model is
specified by 'global' random values and used to generate independent observed
data points; this gives rise to amortized local inference supporting global
model learning.
Learning to Generate Compositional Color Descriptions
Authors: Will Monroe, Noah D. Goodman, Christopher Potts
Date: 2016-06-13 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Language Modeling, Language Modelling, Text Generation

The production of color language is essential for grounded language
generation. Color descriptions have many challenging properties: they can be
vague, compositionally complex, and denotationally rich. We present an
effective approach to generating color descriptions using recurrent neural
networks and a Fourier-transformed color representation. Our model outperforms
previous work on a conditional language modeling task over a large corpus of
naturalistic color descriptions. In addition, probing the model's output
reveals that it can accurately produce not only basic color terms but also
descriptors with non-convex denotations ("greenish"), bare modifiers ("bright",
"dull"), and compositional phrases ("faded teal") not seen in training.
Semantic Decomposition and Recognition of Long and Complex Manipulation Action Sequences
Authors: Eren Erdal Aksoy, Adil Orhan, Florentin Woergoetter
Date: 2016-10-18 00:00:00
Tasks: Semantic Segmentation

Understanding continuous human actions is a non-trivial but important problem
in computer vision. Although there exists a large corpus of work in the
recognition of action sequences, most approaches suffer from problems relating
to vast variations in motions, action combinations, and scene contexts. In this
paper, we introduce a novel method for semantic segmentation and recognition of
long and complex manipulation action tasks, such as "preparing a breakfast" or
"making a sandwich". We represent manipulations with our recently introduced
"Semantic Event Chain" (SEC) concept, which captures the underlying
spatiotemporal structure of an action invariant to motion, velocity, and scene
context. Solely based on the spatiotemporal interactions between manipulated
objects and hands in the extracted SEC, the framework automatically parses
individual manipulation streams performed either sequentially or concurrently.
Using event chains, our method further extracts basic primitive elements of
each parsed manipulation. Without requiring any prior object knowledge, the
proposed framework can also extract object-like scene entities that exhibit the
same role in semantically similar manipulations. We conduct extensive
experiments on various recent datasets to validate the robustness of the
framework.