pwc-archive/papers-with-abstracts huggingface dataset:
Fully Distributed and Asynchronized Stochastic Gradient Descent for Networked Systems
Authors: Ying Zhang
Date: 2017-04-13 00:00:00

This paper considers a general data-fitting problem over a networked system,
in which many computing nodes are connected by an undirected graph. This kind
of problem can find many real-world applications and has been studied
extensively in the literature. However, existing solutions either need a
central controller for information sharing or requires slot synchronization
among different nodes, which increases the difficulty of practical
implementations, especially for a very large and heterogeneous system.
  As a contrast, in this paper, we treat the data-fitting problem over the
network as a stochastic programming problem with many constraints. By adapting
the results in a recent paper, we design a fully distributed and asynchronized
stochastic gradient descent (SGD) algorithm. We show that our algorithm can
achieve global optimality and consensus asymptotically by only local
computations and communications. Additionally, we provide a sharp lower bound
for the convergence speed in the regular graph case. This result fits the
intuition and provides guidance to design a `good' network topology to speed up
the convergence. Also, the merit of our design is validated by experiments on
both synthetic and real-world datasets.
Mobile Keyboard Input Decoding with Finite-State Transducers
Authors: Tom Ouyang, David Rybach, Françoise Beaufays, Michael Riley
Date: 2017-04-13 00:00:00
Tasks: Decoder, speech-recognition, Speech Recognition

We propose a finite-state transducer (FST) representation for the models used
to decode keyboard inputs on mobile devices. Drawing from learnings from the
field of speech recognition, we describe a decoding framework that can satisfy
the strict memory and latency constraints of keyboard input. We extend this
framework to support functionalities typically not present in speech
recognition, such as literal decoding, autocorrections, word completions, and
next word predictions.
  We describe the general framework of what we call for short the keyboard "FST
decoder" as well as the implementation details that are new compared to a
speech FST decoder. We demonstrate that the FST decoder enables new UX features
such as post-corrections. Finally, we sketch how this decoder can support
advanced features such as personalization and contextualization.
Gaussian variational approximation with sparse precision matrices
Authors: Linda S. L. Tan, David J. Nott
Date: 2016-05-18 00:00:00
Tasks: State Space Models, Time Series, Time Series Analysis

We consider the problem of learning a Gaussian variational approximation to
the posterior distribution for a high-dimensional parameter, where we impose
sparsity in the precision matrix to reflect appropriate conditional
independence structure in the model. Incorporating sparsity in the precision
matrix allows the Gaussian variational distribution to be both flexible and
parsimonious, and the sparsity is achieved through parameterization in terms of
the Cholesky factor. Efficient stochastic gradient methods which make
appropriate use of gradient information for the target distribution are
developed for the optimization. We consider alternative estimators of the
stochastic gradients which have lower variation and are more stable. Our
approach is illustrated using generalized linear mixed models and state space
models for time series.
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
Authors: Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, Shin Ishii
Date: 2017-04-13 00:00:00
Tasks: Semi-Supervised Image Classification

We propose a new regularization method based on virtual adversarial loss: a
new measure of local smoothness of the conditional label distribution given
input. Virtual adversarial loss is defined as the robustness of the conditional
label distribution around each input data point against local perturbation.
Unlike adversarial training, our method defines the adversarial direction
without label information and is hence applicable to semi-supervised learning.
Because the directions in which we smooth the model are only "virtually"
adversarial, we call our method virtual adversarial training (VAT). The
computational cost of VAT is relatively low. For neural networks, the
approximated gradient of virtual adversarial loss can be computed with no more
than two pairs of forward- and back-propagations. In our experiments, we
applied VAT to supervised and semi-supervised learning tasks on multiple
benchmark datasets. With a simple enhancement of the algorithm based on the
entropy minimization principle, our VAT achieves state-of-the-art performance
for semi-supervised learning tasks on SVHN and CIFAR-10.
Approximate Kernel-based Conditional Independence Tests for Fast Non-Parametric Causal Discovery
Authors: Eric V. Strobl, Kun Zhang, Shyam Visweswaran
Date: 2017-02-13 00:00:00
Tasks: Causal Discovery

Constraint-based causal discovery (CCD) algorithms require fast and accurate
conditional independence (CI) testing. The Kernel Conditional Independence Test
(KCIT) is currently one of the most popular CI tests in the non-parametric
setting, but many investigators cannot use KCIT with large datasets because the
test scales cubicly with sample size. We therefore devise two relaxations
called the Randomized Conditional Independence Test (RCIT) and the Randomized
conditional Correlation Test (RCoT) which both approximate KCIT by utilizing
random Fourier features. In practice, both of the proposed tests scale linearly
with sample size and return accurate p-values much faster than KCIT in the
large sample size context. CCD algorithms run with RCIT or RCoT also return
graphs at least as accurate as the same algorithms run with KCIT but with large
reductions in run time.
Convergence analysis of the information matrix in Gaussian belief propagation
Authors: Jian Du, Shaodan Ma, Yik-Chung Wu, Soummya Kar, José M. F. Moura
Date: 2017-04-13 00:00:00

Gaussian belief propagation (BP) has been widely used for distributed
estimation in large-scale networks such as the smart grid, communication
networks, and social networks, where local measurements/observations are
scattered over a wide geographical area. However, the convergence of Gaus- sian
BP is still an open issue. In this paper, we consider the convergence of
Gaussian BP, focusing in particular on the convergence of the information
matrix. We show analytically that the exchanged message information matrix
converges for arbitrary positive semidefinite initial value, and its dis- tance
to the unique positive definite limit matrix decreases exponentially fast.
Collaborative Low-Rank Subspace Clustering
Authors: Stephen Tierney, Yi Guo, Junbin Gao
Date: 2017-04-13 00:00:00
Tasks: Clustering

In this paper we present Collaborative Low-Rank Subspace Clustering. Given
multiple observations of a phenomenon we learn a unified representation matrix.
This unified matrix incorporates the features from all the observations, thus
increasing the discriminative power compared with learning the representation
matrix on each observation separately. Experimental evaluation shows that our
method outperforms subspace clustering on separate observations and the state
of the art collaborative learning algorithm.
Tractable Clustering of Data on the Curve Manifold
Authors: Stephen Tierney, Junbin Gao, Yi Guo, Zheng Zhang
Date: 2017-04-13 00:00:00
Tasks: Clustering

In machine learning it is common to interpret each data point as a vector in
Euclidean space. However the data may actually be functional i.e.\ each data
point is a function of some variable such as time and the function is
discretely sampled. The naive treatment of functional data as traditional
multivariate data can lead to poor performance since the algorithms are
ignoring the correlation in the curvature of each function. In this paper we
propose a tractable method to cluster functional data or curves by adapting the
Euclidean Low-Rank Representation (LRR) to the curve manifold. Experimental
evaluation on synthetic and real data reveals that this method massively
outperforms prior clustering methods in both speed and accuracy when clustering
functional data.
Efficient Sparse Subspace Clustering by Nearest Neighbour Filtering
Authors: Stephen Tierney, Yi Guo, Junbin Gao
Date: 2017-04-13 00:00:00
Tasks: Clustering, General Classification

Sparse Subspace Clustering (SSC) has been used extensively for subspace
identification tasks due to its theoretical guarantees and relative ease of
implementation. However SSC has quadratic computation and memory requirements
with respect to the number of input data points. This burden has prohibited
SSCs use for all but the smallest datasets. To overcome this we propose a new
method, k-SSC, that screens out a large number of data points to both reduce
SSC to linear memory and computational requirements. We provide theoretical
analysis for the bounds of success for k-SSC. Our experiments show that k-SSC
exceeds theoretical expectations and outperforms existing SSC approximations by
maintaining the classification performance of SSC. Furthermore in the spirit of
reproducible research we have publicly released the source code for k-SSC
Weak Adaptive Submodularity and Group-Based Active Diagnosis with Applications to State Estimation with Persistent Sensor Faults
Authors: Sze Zheng Yong, Lingyun Gao, Necmiye Ozay
Date: 2017-01-24 00:00:00
Tasks: Decision Making, Medical Diagnosis, State Estimation

In this paper, we consider adaptive decision-making problems for stochastic
state estimation with partial observations. First, we introduce the concept of
weak adaptive submodularity, a generalization of adaptive submodularity, which
has found great success in solving challenging adaptive state estimation
problems. Then, for the problem of active diagnosis, i.e., discrete state
estimation via active sensing, we show that an adaptive greedy policy has a
near-optimal performance guarantee when the reward function possesses this
property. We further show that the reward function for group-based active
diagnosis, which arises in applications such as medical diagnosis and state
estimation with persistent sensor faults, is also weakly adaptive submodular.
Finally, in experiments of state estimation for an aircraft electrical system
with persistent sensor faults, we observe that an adaptive greedy policy
performs equally well as an exhaustive search.
HLA class I binding prediction via convolutional neural networks
Authors: Yeeleng Scott Vang, Xiaohui Xie
Date: 2017-01-03 00:00:00
Tasks: BIG-bench Machine Learning, MHC presentation prediction, Prediction

Many biological processes are governed by protein-ligand interactions. One
such example is the recognition of self and nonself cells by the immune system.
This immune response process is regulated by the major histocompatibility
complex (MHC) protein which is encoded by the human leukocyte antigen (HLA)
complex. Understanding the binding potential between MHC and peptides can lead
to the design of more potent, peptide-based vaccines and immunotherapies for
infectious autoimmune diseases.
  We apply machine learning techniques from the natural language processing
(NLP) domain to address the task of MHC-peptide binding prediction. More
specifically, we introduce a new distributed representation of amino acids,
name HLA-Vec, that can be used for a variety of downstream proteomic machine
learning tasks. We then propose a deep convolutional neural network
architecture, name HLA-CNN, for the task of HLA class I-peptide binding
prediction. Experimental results show combining the new distributed
representation with our HLA-CNN architecture achieves state-of-the-art results
in the majority of the latest two Immune Epitope Database (IEDB) weekly
automated benchmark datasets. We further apply our model to predict binding on
the human genome and identify 15 genes with potential for self binding.
Bag-of-Words Method Applied to Accelerometer Measurements for the Purpose of Classification and Energy Estimation
Authors: Kevin M. Amaral, Ping Chen, Scott Crouter, Wei Ding
Date: 2017-04-05 00:00:00
Tasks: General Classification, regression

Accelerometer measurements are the prime type of sensor information most
think of when seeking to measure physical activity. On the market, there are
many fitness measuring devices which aim to track calories burned and steps
counted through the use of accelerometers. These measurements, though good
enough for the average consumer, are noisy and unreliable in terms of the
precision of measurement needed in a scientific setting. The contribution of
this paper is an innovative and highly accurate regression method which uses an
intermediary two-stage classification step to better direct the regression of
energy expenditure values from accelerometer counts.
  We show that through an additional unsupervised layer of intermediate feature
construction, we can leverage latent patterns within accelerometer counts to
provide better grounds for activity classification than expert-constructed
timeseries features. For this, our approach utilizes a mathematical model
originating in natural language processing, the bag-of-words model, that has in
the past years been appearing in diverse disciplines outside of the natural
language processing field such as image processing. Further emphasizing the
natural language connection to stochastics, we use a gaussian mixture model to
learn the dictionary upon which the bag-of-words model is built. Moreover, we
show that with the addition of these features, we're able to improve regression
root mean-squared error of energy expenditure by approximately 1.4 units over
existing state-of-the-art methods.
Top-down Visual Saliency Guided by Captions
Authors: Vasili Ramanishka, Abir Das, Jianming Zhang, Kate Saenko
Date: 2016-12-21 00:00:00
Proceeding: CVPR 2017 7
Tasks: Decoder, Sentence, Video Captioning

Neural image/video captioning models can generate accurate descriptions, but
their internal process of mapping regions to words is a black box and therefore
difficult to explain. Top-down neural saliency methods can find important
regions given a high-level semantic task such as object classification, but
cannot use a natural language sentence as the top-down input for the task. In
this paper, we propose Caption-Guided Visual Saliency to expose the
region-to-word mapping in modern encoder-decoder networks and demonstrate that
it is learned implicitly from caption training data, without any pixel-level
annotations. Our approach can produce spatial or spatiotemporal heatmaps for
both predicted captions, and for arbitrary query sentences. It recovers
saliency without the overhead of introducing explicit attention layers, and can
be used to analyze a variety of existing model architectures and improve their
design. Evaluation on large-scale video and image datasets demonstrates that
our approach achieves comparable captioning performance with existing methods
while providing more accurate saliency heatmaps. Our code is available at
visionlearninggroup.github.io/caption-guided-saliency/.
Deep-Learning for Classification of Colorectal Polyps on Whole-Slide Images
Authors: Bruno Korbar, Andrea M. Olofson, Allen P. Miraflor, Katherine M. Nicka, Matthew A. Suriawinata, Lorenzo Torresani, Arief A. Suriawinata, Saeed Hassanpour
Date: 2017-03-05 00:00:00
Tasks: General Classification, whole slide images

Histopathological characterization of colorectal polyps is an important
principle for determining the risk of colorectal cancer and future rates of
surveillance for patients. This characterization is time-intensive, requires
years of specialized training, and suffers from significant inter-observer and
intra-observer variability. In this work, we built an automatic
image-understanding method that can accurately classify different types of
colorectal polyps in whole-slide histology images to help pathologists with
histopathological characterization and diagnosis of colorectal polyps. The
proposed image-understanding method is based on deep-learning techniques, which
rely on numerous levels of abstraction for data representation and have shown
state-of-the-art results for various image analysis tasks. Our
image-understanding method covers all five polyp types (hyperplastic polyp,
sessile serrated polyp, traditional serrated adenoma, tubular adenoma, and
tubulovillous/villous adenoma) that are included in the US multi-society task
force guidelines for colorectal cancer risk assessment and surveillance, and
encompasses the most common occurrences of colorectal polyps. Our evaluation on
239 independent test samples shows our proposed method can identify the types
of colorectal polyps in whole-slide images with a high efficacy (accuracy:
93.0%, precision: 89.7%, recall: 88.3%, F1 score: 88.8%). The presented method
in this paper can reduce the cognitive burden on pathologists and improve their
accuracy and efficiency in histopathological characterization of colorectal
polyps, and in subsequent risk assessment and follow-up recommendations.
Asymmetric Feature Maps with Application to Sketch Based Retrieval
Authors: Giorgos Tolias, Ondřej Chum
Date: 2017-04-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Retrieval, Retrieval, Sketch-Based Image Retrieval, Translation

We propose a novel concept of asymmetric feature maps (AFM), which allows to
evaluate multiple kernels between a query and database entries without
increasing the memory requirements. To demonstrate the advantages of the AFM
method, we derive a short vector image representation that, due to asymmetric
feature maps, supports efficient scale and translation invariant sketch-based
image retrieval. Unlike most of the short-code based retrieval systems, the
proposed method provides the query localization in the retrieved image. The
efficiency of the search is boosted by approximating a 2D translation search
via trigonometric polynomial of scores by 1D projections. The projections are a
special case of AFM. An order of magnitude speed-up is achieved compared to
traditional trigonometric polynomials. The results are boosted by an
image-based average query expansion, exceeding significantly the state of the
art on standard benchmarks.
Beyond Uniform Priors in Bayesian Network Structure Learning
Authors: Marco Scutari
Date: 2017-04-12 00:00:00

Bayesian network structure learning is often performed in a Bayesian setting,
evaluating candidate structures using their posterior probabilities for a given
data set. Score-based algorithms then use those posterior probabilities as an
objective function and return the maximum a posteriori network as the learned
model. For discrete Bayesian networks, the canonical choice for a posterior
score is the Bayesian Dirichlet equivalent uniform (BDeu) marginal likelihood
with a uniform (U) graph prior, which assumes a uniform prior both on the
network structures and on the parameters of the networks. In this paper, we
investigate the problems arising from these assumptions, focusing on those
caused by small sample sizes and sparse data. We then propose an alternative
posterior score: the Bayesian Dirichlet sparse (BDs) marginal likelihood with a
marginal uniform (MU) graph prior. Like U+BDeu, MU+BDs does not require any
prior information on the probabilistic structure of the data and can be used as
a replacement noninformative score. We study its theoretical properties and we
evaluate its performance in an extensive simulation study, showing that MU+BDs
is both more accurate than U+BDeu in learning the structure of the network and
competitive in predicting power, while not being computationally more complex
to estimate.
Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysis
Authors: Haohan Wang, Aaksha Meghawat, Louis-Philippe Morency, Eric P. Xing
Date: 2016-09-16 00:00:00
Tasks: Multimodal Sentiment Analysis, Sentiment Analysis, Sentiment Classification

Multimodal sentiment analysis is drawing an increasing amount of attention
these days. It enables mining of opinions in video reviews which are now
available aplenty on online platforms. However, multimodal sentiment analysis
has only a few high-quality data sets annotated for training machine learning
algorithms. These limited resources restrict the generalizability of models,
where, for example, the unique characteristics of a few speakers (e.g., wearing
glasses) may become a confounding factor for the sentiment classification task.
In this paper, we propose a Select-Additive Learning (SAL) procedure that
improves the generalizability of trained neural networks for multimodal
sentiment analysis. In our experiments, we show that our SAL approach improves
prediction accuracy significantly in all three modalities (verbal, acoustic,
visual), as well as in their fusion. Our results show that SAL, even when
trained on one dataset, achieves good generalization across two new test
datasets.
Provable Self-Representation Based Outlier Detection in a Union of Subspaces
Authors: Chong You, Daniel P. Robinson, René Vidal
Date: 2017-04-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: Outlier Detection

Many computer vision tasks involve processing large amounts of data
contaminated by outliers, which need to be detected and rejected. While outlier
detection methods based on robust statistics have existed for decades, only
recently have methods based on sparse and low-rank representation been
developed along with guarantees of correct outlier detection when the inliers
lie in one or more low-dimensional subspaces. This paper proposes a new outlier
detection method that combines tools from sparse representation with random
walks on a graph. By exploiting the property that data points can be expressed
as sparse linear combinations of each other, we obtain an asymmetric affinity
matrix among data points, which we use to construct a weighted directed graph.
By defining a suitable Markov Chain from this graph, we establish a connection
between inliers/outliers and essential/inessential states of the Markov chain,
which allows us to detect outliers by using random walks. We provide a
theoretical analysis that justifies the correctness of our method under
geometric and connectivity assumptions. Experimental results on image databases
demonstrate its superiority with respect to state-of-the-art sparse and
low-rank outlier detection methods.
Alignment-based compositional semantics for instruction following
Authors: Jacob Andreas, Dan Klein
Date: 2015-08-26 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Instruction Following, Sentence

This paper describes an alignment-based model for interpreting natural
language instructions in context. We approach instruction following as a search
over plans, scoring sequences of actions conditioned on structured observations
of text and the environment. By explicitly modeling both the low-level
compositional structure of individual actions and the high-level structure of
full plans, we are able to learn both grounded representations of sentence
meaning and pragmatic constraints on interpretation. To demonstrate the model's
flexibility, we apply it to a diverse set of benchmark tasks. On every task, we
outperform strong task-specific baselines, and achieve several new
state-of-the-art results.