pwc-archive/papers-with-abstracts huggingface dataset:
Active Learning for Speech Recognition: the Power of Gradients
Authors: Jiaji Huang, Rewon Child, Vinay Rao, Hairong Liu, Sanjeev Satheesh, Adam Coates
Date: 2016-12-10 00:00:00
Tasks: Active Learning, Informativeness, speech-recognition, Speech Recognition

In training speech recognition systems, labeling audio clips can be
expensive, and not all data is equally valuable. Active learning aims to label
only the most informative samples to reduce cost. For speech recognition,
confidence scores and other likelihood-based active learning methods have been
shown to be effective. Gradient-based active learning methods, however, are
still not well-understood. This work investigates the Expected Gradient Length
(EGL) approach in active learning for end-to-end speech recognition. We justify
EGL from a variance reduction perspective, and observe that EGL's measure of
informativeness picks novel samples uncorrelated with confidence scores.
Experimentally, we show that EGL can reduce word errors by 11\%, or
alternatively, reduce the number of samples to label by 50\%, when compared to
random sampling.
Automatic Lymphocyte Detection in H&E Images with Deep Neural Networks
Authors: Jianxu Chen, Chukka Srinivas
Date: 2016-12-09 00:00:00

Automatic detection of lymphocyte in H&E images is a necessary first step in
lots of tissue image analysis algorithms. An accurate and robust automated
lymphocyte detection approach is of great importance in both computer science
and clinical studies. Most of the existing approaches for lymphocyte detection
are based on traditional image processing algorithms and/or classic machine
learning methods. In the recent years, deep learning techniques have
fundamentally transformed the way that a computer interprets images and have
become a matchless solution in various pattern recognition problems. In this
work, we design a new deep neural network model which extends the fully
convolutional network by combining the ideas in several recent techniques, such
as shortcut links. Also, we design a new training scheme taking the prior
knowledge about lymphocytes into consideration. The training scheme not only
efficiently exploits the limited amount of free-form annotations from
pathologists, but also naturally supports efficient fine-tuning. As a
consequence, our model has the potential of self-improvement by leveraging the
errors collected during real applications. Our experiments show that our deep
neural network model achieves good performance in the images of different
staining conditions or different types of tissues.
Towards deep learning with spiking neurons in energy based models with contrastive Hebbian plasticity
Authors: Thomas Mesnard, Wulfram Gerstner, Johanni Brea
Date: 2016-12-09 00:00:00
Tasks: Deep Learning, General Classification, regression, Reinforcement Learning, Reinforcement Learning (RL)

In machine learning, error back-propagation in multi-layer neural networks
(deep learning) has been impressively successful in supervised and
reinforcement learning tasks. As a model for learning in the brain, however,
deep learning has long been regarded as implausible, since it relies in its
basic form on a non-local plasticity rule. To overcome this problem,
energy-based models with local contrastive Hebbian learning were proposed and
tested on a classification task with networks of rate neurons. We extended this
work by implementing and testing such a model with networks of leaky
integrate-and-fire neurons. Preliminary results indicate that it is possible to
learn a non-linear regression task with hidden layers, spiking neurons and a
local synaptic plasticity rule.
Evaluating Creative Language Generation: The Case of Rap Lyric Ghostwriting
Authors: Peter Potash, Alexey Romanov, Anna Rumshisky
Date: 2016-12-09 00:00:00
Proceeding: WS 2018 6
Tasks: Text Generation

Language generation tasks that seek to mimic human ability to use language
creatively are difficult to evaluate, since one must consider creativity,
style, and other non-trivial aspects of the generated text. The goal of this
paper is to develop evaluation methods for one such task, ghostwriting of rap
lyrics, and to provide an explicit, quantifiable foundation for the goals and
future directions of this task. Ghostwriting must produce text that is similar
in style to the emulated artist, yet distinct in content. We develop a novel
evaluation methodology that addresses several complementary aspects of this
task, and illustrate how such evaluation can be used to meaningfully analyze
system performance. We provide a corpus of lyrics for 13 rap artists, annotated
for stylistic similarity, which allows us to assess the feasibility of manual
evaluation for generated verse.
PVANet: Lightweight Deep Neural Networks for Real-time Object Detection
Authors: Sanghoon Hong, Byungseok Roh, Kye-Hyeon Kim, Yeongjae Cheon, Minje Park
Date: 2016-11-23 00:00:00
Tasks: Object, object-detection, Object Detection, Real-Time Object Detection

In object detection, reducing computational cost is as important as improving
accuracy for most practical usages. This paper proposes a novel network
structure, which is an order of magnitude lighter than other state-of-the-art
networks while maintaining the accuracy. Based on the basic principle of more
layers with less channels, this new deep neural network minimizes its
redundancy by adopting recent innovations including C.ReLU and Inception
structure. We also show that this network can be trained efficiently to achieve
solid results on well-known object detection benchmarks: 84.9% and 84.2% mAP on
VOC2007 and VOC2012 while the required compute is less than 10% of the recent
ResNet-101.
On Tie Strength Augmented Social Correlation for Inferring Preference of Mobile Telco Users
Authors: Shifeng Liu, Zheng Hu, Sujit Dey, Xin Ke
Date: 2016-03-01 00:00:00
Tasks: Collaborative Filtering, Marketing

For mobile telecom operators, it is critical to build preference profiles of
their customers and connected users, which can help operators make better
marketing strategies, and provide more personalized services. With the
deployment of deep packet inspection (DPI) in telecom networks, it is possible
for the telco operators to obtain user online preference. However, DPI has its
limitations and user preference derived only from DPI faces sparsity and cold
start problems. To better infer the user preference, social correlation in
telco users network derived from Call Detailed Records (CDRs) with regard to
online preference is investigated. Though widely verified in several online
social networks, social correlation between online preference of users in
mobile telco networks, where the CDRs derived relationship are of less social
properties and user mobile internet surfing activities are not visible to
neighbourhood, has not been explored at a large scale. Based on a real world
telecom dataset including CDRs and preference of more than $550K$ users for
several months, we verified that correlation does exist between online
preference in such \textit{ambiguous} social network. Furthermore, we found
that the stronger ties that users build, the more similarity between their
preference may have. After defining the preference inferring task as a Top-$K$
recommendation problem, we incorporated Matrix Factorization Collaborative
Filtering model with social correlation and tie strength based on call patterns
to generate Top-$K$ preferred categories for users. The proposed Tie Strength
Augmented Social Recommendation (TSASoRec) model takes data sparsity and cold
start user problems into account, considering both the recorded and missing
recorded category entries. The experiment on real dataset shows the proposed
model can better infer user preference, especially for cold start users.
Square Hellinger Subadditivity for Bayesian Networks and its Applications to Identity Testing
Authors: Constantinos Daskalakis, Qinxuan Pan
Date: 2016-12-09 00:00:00

We show that the square Hellinger distance between two Bayesian networks on
the same directed graph, $G$, is subadditive with respect to the neighborhoods
of $G$. Namely, if $P$ and $Q$ are the probability distributions defined by two
Bayesian networks on the same DAG, our inequality states that the square
Hellinger distance, $H^2(P,Q)$, between $P$ and $Q$ is upper bounded by the
sum, $\sum_v H^2(P_{\{v\} \cup \Pi_v}, Q_{\{v\} \cup \Pi_v})$, of the square
Hellinger distances between the marginals of $P$ and $Q$ on every node $v$ and
its parents $\Pi_v$ in the DAG. Importantly, our bound does not involve the
conditionals but the marginals of $P$ and $Q$. We derive a similar inequality
for more general Markov Random Fields.
  As an application of our inequality, we show that distinguishing whether two
Bayesian networks $P$ and $Q$ on the same (but potentially unknown) DAG satisfy
$P=Q$ vs $d_{\rm TV}(P,Q)>\epsilon$ can be performed from
$\tilde{O}(|\Sigma|^{3/4(d+1)} \cdot n/\epsilon^2)$ samples, where $d$ is the
maximum in-degree of the DAG and $\Sigma$ the domain of each variable of the
Bayesian networks. If $P$ and $Q$ are defined on potentially different and
potentially unknown trees, the sample complexity becomes
$\tilde{O}(|\Sigma|^{4.5} n/\epsilon^2)$, whose dependence on $n, \epsilon$ is
optimal up to logarithmic factors. Lastly, if $P$ and $Q$ are product
distributions over $\{0,1\}^n$ and $Q$ is known, the sample complexity becomes
$O(\sqrt{n}/\epsilon^2)$, which is optimal up to constant factors.
Testing Bayesian Networks
Authors: Clement Canonne, Ilias Diakonikolas, Daniel Kane, Alistair Stewart
Date: 2016-12-09 00:00:00

This work initiates a systematic investigation of testing high-dimensional structured distributions by focusing on testing Bayesian networks -- the prototypical family of directed graphical models. A Bayesian network is defined by a directed acyclic graph, where we associate a random variable with each node. The value at any particular node is conditionally independent of all the other non-descendant nodes once its parents are fixed. Specifically, we study the properties of identity testing and closeness testing of Bayesian networks. Our main contribution is the first non-trivial efficient testing algorithms for these problems and corresponding information-theoretic lower bounds. For a wide range of parameter settings, our testing algorithms have sample complexity sublinear in the dimension and are sample-optimal, up to constant factors.
Panoptic Studio: A Massively Multiview System for Social Interaction Capture
Authors: Hanbyul Joo, Tomas Simon, Xulong Li, Hao liu, Lei Tan, Lin Gui, Sean Banerjee, Timothy Godisart, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, Yaser Sheikh
Date: 2016-12-09 00:00:00

We present an approach to capture the 3D motion of a group of people engaged
in a social interaction. The core challenges in capturing social interactions
are: (1) occlusion is functional and frequent; (2) subtle motion needs to be
measured over a space large enough to host a social group; (3) human appearance
and configuration variation is immense; and (4) attaching markers to the body
may prime the nature of interactions. The Panoptic Studio is a system organized
around the thesis that social interactions should be measured through the
integration of perceptual analyses over a large variety of view points. We
present a modularized system designed around this principle, consisting of
integrated structural, hardware, and software innovations. The system takes, as
input, 480 synchronized video streams of multiple people engaged in social
activities, and produces, as output, the labeled time-varying 3D structure of
anatomical landmarks on individuals in the space. Our algorithm is designed to
fuse the "weak" perceptual processes in the large number of views by
progressively generating skeletal proposals from low-level appearance cues, and
a framework for temporal refinement is also presented by associating body parts
to reconstructed dense 3D trajectory stream. Our system and method are the
first in reconstructing full body motion of more than five people engaged in
social interactions without using markers. We also empirically demonstrate the
impact of the number of views in achieving this goal.
Optimal mean-based algorithms for trace reconstruction
Authors: Anindya De, Ryan O'Donnell, Rocco Servedio
Date: 2016-12-09 00:00:00

In the (deletion-channel) trace reconstruction problem, there is an unknown
$n$-bit source string $x$. An algorithm is given access to independent traces
of $x$, where a trace is formed by deleting each bit of~$x$ independently with
probability~$\delta$. The goal of the algorithm is to recover~$x$ exactly (with
high probability), while minimizing samples (number of traces) and running
time.
  Previously, the best known algorithm for the trace reconstruction problem was
due to Holenstein~et~al.; it uses $\exp(\tilde{O}(n^{1/2}))$ samples and
running time for any fixed $0 < \delta < 1$. It is also what we call a
"mean-based algorithm", meaning that it only uses the empirical means of the
individual bits of the traces. Holenstein~et~al.~also gave a lower bound,
showing that any mean-based algorithm must use at least $n^{\tilde{\Omega}(\log
n)}$ samples.
  In this paper we improve both of these results, obtaining matching upper and
lower bounds for mean-based trace reconstruction. For any constant deletion
rate $0 < \delta < 1$, we give a mean-based algorithm that uses
$\exp(O(n^{1/3}))$ time and traces; we also prove that any mean-based algorithm
must use at least $\exp(\Omega(n^{1/3}))$ traces. In fact, we obtain matching
upper and lower bounds even for $\delta$ subconstant and $\rho := 1-\delta$
subconstant: when $(\log^3 n)/n \ll \delta \leq 1/2$ the bound is
$\exp(-\Theta(\delta n)^{1/3})$, and when $1/\sqrt{n} \ll \rho \leq 1/2$ the
bound is $\exp(-\Theta(n/\rho)^{1/3})$.
  Our proofs involve estimates for the maxima of Littlewood polynomials on
complex disks. We show that these techniques can also be used to perform trace
reconstruction with random insertions and bit-flips in addition to deletions.
We also find a surprising result: for deletion probabilities $\delta > 1/2$,
the presence of insertions can actually help with trace reconstruction.
Robust mixture of experts modeling using the skew $t$ distribution
Authors: Faicel Chamroukhi
Date: 2016-12-09 00:00:00
Tasks: Clustering, Mixture-of-Experts, regression

Mixture of Experts (MoE) is a popular framework in the fields of statistics
and machine learning for modeling heterogeneity in data for regression,
classification and clustering. MoE for continuous data are usually based on the
normal distribution. However, it is known that for data with asymmetric
behavior, heavy tails and atypical observations, the use of the normal
distribution is unsuitable. We introduce a new robust non-normal mixture of
experts modeling using the skew $t$ distribution. The proposed skew $t$ mixture
of experts, named STMoE, handles these issues of the normal mixtures experts
regarding possibly skewed, heavy-tailed and noisy data. We develop a dedicated
expectation conditional maximization (ECM) algorithm to estimate the model
parameters by monotonically maximizing the observed data log-likelihood. We
describe how the presented model can be used in prediction and in model-based
clustering of regression data. Numerical experiments carried out on simulated
data show the effectiveness and the robustness of the proposed model in fitting
non-linear regression functions as well as in model-based clustering. Then, the
proposed model is applied to the real-world data of tone perception for musical
data analysis, and the one of temperature anomalies for the analysis of climate
change data. The obtained results confirm the usefulness of the model for
practical data analysis applications.
SimTensor: A synthetic tensor data generator
Authors: Hadi Fanaee-T, Joao Gama
Date: 2016-12-09 00:00:00

SimTensor is a multi-platform, open-source software for generating artificial
tensor data (either with CP/PARAFAC or Tucker structure) for reproducible
research on tensor factorization algorithms. SimTensor is a stand-alone
application based on MATALB. It provides a wide range of facilities for
generating tensor data with various configurations. It comes with a
user-friendly graphical user interface, which enables the user to generate
tensors with complicated settings in an easy way. It also has this facility to
export generated data to universal formats such as CSV and HDF5, which can be
imported via a wide range of programming languages (C, C++, Java, R, Fortran,
MATLAB, Perl, Python, and many more). The most innovative part of SimTensor is
this that can generate temporal tensors with periodic waves, seasonal effects
and streaming structure. it can apply constraints such as non-negativity and
different kinds of sparsity to the data. SimTensor also provides this facility
to simulate different kinds of change-points and inject various types of
anomalies. The source code and binary versions of SimTensor is available for
download in http://www.simtensor.org.
Advancing Bayesian Optimization: The Mixed-Global-Local (MGL) Kernel and Length-Scale Cool Down
Authors: Kim Peter Wabersich, Marc Toussaint
Date: 2016-12-09 00:00:00
Tasks: Bayesian Optimization, global-optimization

Bayesian Optimization (BO) has become a core method for solving expensive
black-box optimization problems. While much research focussed on the choice of
the acquisition function, we focus on online length-scale adaption and the
choice of kernel function. Instead of choosing hyperparameters in view of
maximum likelihood on past data, we propose to use the acquisition function to
decide on hyperparameter adaptation more robustly and in view of the future
optimization progress. Further, we propose a particular kernel function that
includes non-stationarity and local anisotropy and thereby implicitly
integrates the efficiency of local convex optimization with global Bayesian
optimization. Comparisons to state-of-the art BO methods underline the
efficiency of these mechanisms on global optimization benchmarks.
Following Gaze Across Views
Authors: Adrià Recasens, Carl Vondrick, Aditya Khosla, Antonio Torralba
Date: 2016-12-09 00:00:00

Following the gaze of people inside videos is an important signal for
understanding people and their actions. In this paper, we present an approach
for following gaze across views by predicting where a particular person is
looking throughout a scene. We collect VideoGaze, a new dataset which we use as
a benchmark to both train and evaluate models. Given one view with a person in
it and a second view of the scene, our model estimates a density for gaze
location in the second view. A key aspect of our approach is an end-to-end
model that solves the following sub-problems: saliency, gaze pose, and
geometric relationships between views. Although our model is supervised only
with gaze, we show that the model learns to solve these subproblems
automatically without supervision. Experiments suggest that our approach
follows gaze better than standard baselines and produces plausible results for
everyday situations.
Measuring Adverse Drug Effects on Multimorbity using Tractable Bayesian Networks
Authors: Jessa Bekker, Arjen Hommersom, Martijn Lappenschaar, Jesse Davis
Date: 2016-12-09 00:00:00

Managing patients with multimorbidity often results in polypharmacy: the
prescription of multiple drugs. However, the long-term effects of specific
combinations of drugs and diseases are typically unknown. In particular, drugs
prescribed for one condition may result in adverse effects for the other. To
investigate which types of drugs may affect the further progression of
multimorbidity, we query models of diseases and prescriptions that are learned
from primary care data. State-of-the-art tractable Bayesian network
representations, on which such complex queries can be computed efficiently, are
employed for these large medical networks. Our results confirm that
prescriptions may lead to unintended negative consequences in further
development of multimorbidity in cardiovascular diseases. Moreover, a drug
treatment for one disease group may affect diseases of another group.
Robust mixture of experts modeling using the $t$ distribution
Authors: Faicel Chamroukhi
Date: 2016-12-09 00:00:00
Tasks: Clustering, Mixture-of-Experts, regression

Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in
data for regression, classification, and clustering. For regression and cluster
analyses of continuous data, MoE usually use normal experts following the
Gaussian distribution. However, for a set of data containing a group or groups
of observations with heavy tails or atypical observations, the use of normal
experts is unsuitable and can unduly affect the fit of the MoE model. We
introduce a robust MoE modeling using the $t$ distribution. The proposed $t$
MoE (TMoE) deals with these issues regarding heavy-tailed and noisy data. We
develop a dedicated expectation-maximization (EM) algorithm to estimate the
parameters of the proposed model by monotonically maximizing the observed data
log-likelihood. We describe how the presented model can be used in prediction
and in model-based clustering of regression data. The proposed model is
validated on numerical experiments carried out on simulated data, which show
the effectiveness and the robustness of the proposed model in terms of modeling
non-linear regression functions as well as in model-based clustering. Then, it
is applied to the real-world data of tone perception for musical data analysis,
and the one of temperature anomalies for the analysis of climate change data.
The obtained results show the usefulness of the TMoE model for practical
applications.
Correlated and Individual Multi-Modal Deep Learning for RGB-D Object Recognition
Authors: Ziyan Wang, Jiwen Lu, Ruogu Lin, Jianjiang Feng, Jie zhou
Date: 2016-04-06 00:00:00
Tasks: Object, Object Recognition

In this paper, we propose a new correlated and individual multi-modal deep
learning (CIMDL) method for RGB-D object recognition. Unlike most conventional
RGB-D object recognition methods which extract features from the RGB and depth
channels individually, our CIMDL jointly learns feature representations from
raw RGB-D data with a pair of deep neural networks, so that the sharable and
modal-specific information can be simultaneously exploited. Specifically, we
construct a pair of deep convolutional neural networks (CNNs) for the RGB and
depth data, and concatenate them at the top layer of the network with a loss
function which learns a new feature space where both correlated part and the
individual part of the RGB-D information are well modelled. The parameters of
the whole networks are updated by using the back-propagation criterion.
Experimental results on two widely used RGB-D object image benchmark datasets
clearly show that our method outperforms state-of-the-arts.
High-dimensional regression over disease subgroups
Authors: Frank Dondelinger, Sach Mukherjee, the Alzheimer's Disease Neuroimaging Initiative
Date: 2016-11-03 00:00:00
Tasks: regression, Vocal Bursts Intensity Prediction

We consider high-dimensional regression over subgroups of observations. Our
work is motivated by biomedical problems, where disease subtypes, for example,
may differ with respect to underlying regression models, but sample sizes at
the subgroup-level may be limited. We focus on the case in which
subgroup-specific models may be expected to be similar but not necessarily
identical. Our approach is to treat subgroups as related problem instances and
jointly estimate subgroup-specific regression coefficients. This is done in a
penalized framework, combining an $\ell_1$ term with an additional term that
penalizes differences between subgroup-specific coefficients. This gives
solutions that are globally sparse but that allow information-sharing between
the subgroups. We present algorithms for estimation and empirical results on
simulated data and using Alzheimer's disease, amyotrophic lateral sclerosis and
cancer datasets. These examples demonstrate the gains our approach can offer in
terms of prediction and the ability to estimate subgroup-specific sparsity
patterns.