pwc-archive/papers-with-abstracts huggingface dataset:
Low-Precision Batch-Normalized Activations
Authors: Benjamin Graham
Date: 2017-02-27 00:00:00
Tasks: Quantization

Artificial neural networks can be trained with relatively low-precision
floating-point and fixed-point arithmetic, using between one and 16 bits.
Previous works have focused on relatively wide-but-shallow, feed-forward
networks. We introduce a quantization scheme that is compatible with training
very deep neural networks. Quantizing the network activations in the middle of
each batch-normalization module can greatly reduce the amount of memory and
computational power needed, with little loss in accuracy.
Learning Rates for Kernel-Based Expectile Regression
Authors: Muhammad Farooq, Ingo Steinwart
Date: 2017-02-24 00:00:00
Tasks: regression

Conditional expectiles are becoming an increasingly important tool in finance
as well as in other areas of applications. We analyse a support vector machine
type approach for estimating conditional expectiles and establish learning
rates that are minimax optimal modulo a logarithmic factor if Gaussian RBF
kernels are used and the desired expectile is smooth in a Besov sense. As a
special case, our learning rates improve the best known rates for kernel-based
least squares regression in this scenario. Key ingredients of our statistical
analysis are a general calibration inequality for the asymmetric least squares
loss, a corresponding variance bound as well as an improved entropy number
bound for Gaussian RBF kernels.
Synergistic Team Composition
Authors: Ewa Andrejczuk, Juan A. Rodriguez-Aguilar, Carme Roig, Carles Sierra
Date: 2017-02-27 00:00:00

Effective teams are crucial for organisations, especially in environments
that require teams to be constantly created and dismantled, such as software
development, scientific experiments, crowd-sourcing, or the classroom. Key
factors influencing team performance are competences and personality of team
members. Hence, we present a computational model to compose proficient and
congenial teams based on individuals' personalities and their competences to
perform tasks of different nature. With this purpose, we extend Wilde's
post-Jungian method for team composition, which solely employs individuals'
personalities. The aim of this study is to create a model to partition agents
into teams that are balanced in competences, personality and gender. Finally,
we present some preliminary empirical results that we obtained when analysing
student performance. Results show the benefits of a more informed team
composition that exploits individuals' competences besides information about
their personalities.
A case study on English-Malayalam Machine Translation
Authors: Sreelekha. S, Pushpak Bhattacharyya
Date: 2017-02-27 00:00:00
Tasks: Machine Translation, Translation

In this paper we present our work on a case study on Statistical Machine
Translation (SMT) and Rule based machine translation (RBMT) for translation
from English to Malayalam and Malayalam to English. One of the motivations of
our study is to make a three way performance comparison, such as, a) SMT and
RBMT b) English to Malayalam SMT and Malayalam to English SMT c) English to
Malayalam RBMT and Malayalam to English RBMT. We describe the development of
English to Malayalam and Malayalam to English baseline phrase based SMT system
and the evaluation of its performance compared against the RBMT system. Based
on our study the observations are: a) SMT systems outperform RBMT systems, b)
In the case of SMT, English - Malayalam systems perform better than that of
Malayalam - English systems, c) In the case RBMT, Malayalam to English systems
are performing better than English to Malayalam systems. Based on our
evaluations and detailed error analysis, we describe the requirements of
incorporating morphological processing into the SMT to improve the accuracy of
translation.
Anticipating many futures: Online human motion prediction and synthesis for human-robot collaboration
Authors: Judith Bütepage, Hedvig Kjellström, Danica Kragic
Date: 2017-02-27 00:00:00
Tasks: Human motion prediction, motion prediction

Fluent and safe interactions of humans and robots require both partners to
anticipate the others' actions. A common approach to human intention inference
is to model specific trajectories towards known goals with supervised
classifiers. However, these approaches do not take possible future movements
into account nor do they make use of kinematic cues, such as legible and
predictable motion. The bottleneck of these methods is the lack of an accurate
model of general human motion. In this work, we present a conditional
variational autoencoder that is trained to predict a window of future human
motion given a window of past frames. Using skeletal data obtained from RGB
depth images, we show how this unsupervised approach can be used for online
motion prediction for up to 1660 ms. Additionally, we demonstrate online target
prediction within the first 300-500 ms after motion onset without the use of
target specific training data. The advantage of our probabilistic approach is
the possibility to draw samples of possible future motions. Finally, we
investigate how movements and kinematic cues are represented on the learned low
dimensional manifold.
DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy
Authors: Christian Wachinger, Martin Reuter, Tassilo Klein
Date: 2017-02-27 00:00:00
Tasks: Brain Segmentation, Multi-class Classification, Multi-Task Learning, Segmentation

We introduce DeepNAT, a 3D Deep convolutional neural network for the
automatic segmentation of NeuroAnaTomy in T1-weighted magnetic resonance
images. DeepNAT is an end-to-end learning-based approach to brain segmentation
that jointly learns an abstract feature representation and a multi-class
classification. We propose a 3D patch-based approach, where we do not only
predict the center voxel of the patch but also neighbors, which is formulated
as multi-task learning. To address a class imbalance problem, we arrange two
networks hierarchically, where the first one separates foreground from
background, and the second one identifies 25 brain structures on the
foreground. Since patches lack spatial context, we augment them with
coordinates. To this end, we introduce a novel intrinsic parameterization of
the brain volume, formed by eigenfunctions of the Laplace-Beltrami operator. As
network architecture, we use three convolutional layers with pooling, batch
normalization, and non-linearities, followed by fully connected layers with
dropout. The final segmentation is inferred from the probabilistic output of
the network with a 3D fully connected conditional random field, which ensures
label agreement between close voxels. The roughly 2.7 million parameters in the
network are learned with stochastic gradient descent. Our results show that
DeepNAT compares favorably to state-of-the-art methods. Finally, the purely
learning-based method may have a high potential for the adaptation to young,
old, or diseased brains by fine-tuning the pre-trained network with a small
training sample on the target application, where the availability of larger
datasets with manual annotations may boost the overall segmentation accuracy in
the future.
An update on statistical boosting in biomedicine
Authors: Andreas Mayr, Benjamin Hofner, Elisabeth Waldmann, Tobias Hepp, Olaf Gefeller, Matthias Schmid
Date: 2017-02-27 00:00:00
Tasks: regression, Variable Selection

Statistical boosting algorithms have triggered a lot of research during the
last decade. They combine a powerful machine-learning approach with classical
statistical modelling, offering various practical advantages like automated
variable selection and implicit regularization of effect estimates. They are
extremely flexible, as the underlying base-learners (regression functions
defining the type of effect for the explanatory variables) can be combined with
any kind of loss function (target function to be optimized, defining the type
of regression setting). In this review article, we highlight the most recent
methodological developments on statistical boosting regarding variable
selection, functional regression and advanced time-to-event modelling.
Additionally, we provide a short overview on relevant applications of
statistical boosting in biomedicine.
Fixed-point optimization of deep neural networks with adaptive step size retraining
Authors: Sungho Shin, Yoonho Boo, Wonyong Sung
Date: 2017-02-27 00:00:00
Tasks: Quantization

Fixed-point optimization of deep neural networks plays an important role in
hardware based design and low-power implementations. Many deep neural networks
show fairly good performance even with 2- or 3-bit precision when quantized
weights are fine-tuned by retraining. We propose an improved fixedpoint
optimization algorithm that estimates the quantization step size dynamically
during the retraining. In addition, a gradual quantization scheme is also
tested, which sequentially applies fixed-point optimizations from high- to
low-precision. The experiments are conducted for feed-forward deep neural
networks (FFDNNs), convolutional neural networks (CNNs), and recurrent neural
networks (RNNs).
Communication-efficient Algorithms for Distributed Stochastic Principal Component Analysis
Authors: Dan Garber, Ohad Shamir, Nathan Srebro
Date: 2017-02-27 00:00:00
Proceeding: ICML 2017 8

We study the fundamental problem of Principal Component Analysis in a
statistical distributed setting in which each machine out of $m$ stores a
sample of $n$ points sampled i.i.d. from a single unknown distribution. We
study algorithms for estimating the leading principal component of the
population covariance matrix that are both communication-efficient and achieve
estimation error of the order of the centralized ERM solution that uses all
$mn$ samples. On the negative side, we show that in contrast to results
obtained for distributed estimation under convexity assumptions, for the PCA
objective, simply averaging the local ERM solutions cannot guarantee error that
is consistent with the centralized ERM. We show that this unfortunate phenomena
can be remedied by performing a simple correction step which correlates between
the individual solutions, and provides an estimator that is consistent with the
centralized ERM for sufficiently-large $n$. We also introduce an iterative
distributed algorithm that is applicable in any regime of $n$, which is based
on distributed matrix-vector products. The algorithm gives significant
acceleration in terms of communication rounds over previous distributed
algorithms, in a wide regime of parameters.
Multi-scale Image Fusion Between Pre-operative Clinical CT and X-ray Microtomography of Lung Pathology
Authors: Holger R. Roth, Kai Nagara, Hirohisa ODA, Masahiro Oda, Tomoshi Sugiyama, Shota NAKAMURA, Kensaku MORI
Date: 2017-02-27 00:00:00
Tasks: Anatomy, Computed Tomography (CT), Medical Image Analysis

Computational anatomy allows the quantitative analysis of organs in medical
images. However, most analysis is constrained to the millimeter scale because
of the limited resolution of clinical computed tomography (CT). X-ray
microtomography ($\mu$CT) on the other hand allows imaging of ex-vivo tissues
at a resolution of tens of microns. In this work, we use clinical CT to image
lung cancer patients before partial pneumonectomy (resection of pathological
lung tissue). The resected specimen is prepared for $\mu$CT imaging at a voxel
resolution of 50 $\mu$m (0.05 mm). This high-resolution image of the lung
cancer tissue allows further insides into understanding of tumor growth and
categorization. For making full use of this additional information, image
fusion (registration) needs to be performed in order to re-align the $\mu$CT
image with clinical CT. We developed a multi-scale non-rigid registration
approach. After manual initialization using a few landmark points and rigid
alignment, several levels of non-rigid registration between down-sampled (in
the case of $\mu$CT) and up-sampled (in the case of clinical CT)
representations of the image are performed. Any non-lung tissue is ignored
during the computation of the similarity measure used to guide the registration
during optimization. We are able to recover the volume differences introduced
by the resection and preparation of the lung specimen. The average ($\pm$ std.
dev.) minimum surface distance between $\mu$CT and clinical CT at the resected
lung surface is reduced from 3.3 $\pm$ 2.9 (range: [0.1, 15.9]) to 2.3 mm $\pm$
2.8 (range: [0.0, 15.3]) mm. The alignment of clinical CT with $\mu$CT will
allow further registration with even finer resolutions of $\mu$CT (up to 10
$\mu$m resolution) and ultimately with histopathological microscopy images for
further macro to micro image fusion that can aid medical image analysis.
Learning in Implicit Generative Models
Authors: Shakir Mohamed, Balaji Lakshminarayanan
Date: 2016-10-11 00:00:00
Tasks: Density Ratio Estimation, Econometrics, Two-sample testing

Generative adversarial networks (GANs) provide an algorithmic framework for
constructing generative models with several appealing properties: they do not
require a likelihood function to be specified, only a generating procedure;
they provide samples that are sharp and compelling; and they allow us to
harness our knowledge of building highly accurate neural network classifiers.
Here, we develop our understanding of GANs with the aim of forming a rich view
of this growing area of machine learning---to build connections to the diverse
set of statistical thinking on this topic, of which much can be gained by a
mutual exchange of ideas. We frame GANs within the wider landscape of
algorithms for learning in implicit generative models--models that only specify
a stochastic procedure with which to generate data--and relate these ideas to
modelling problems in related fields, such as econometrics and approximate
Bayesian computation. We develop likelihood-free inference methods and
highlight hypothesis testing as a principle for learning in implicit generative
models, using which we are able to derive the objective function used by GANs,
and many other related objectives. The testing viewpoint directs our focus to
the general problem of density ratio estimation. There are four approaches for
density ratio estimation, one of which is a solution using classifiers to
distinguish real from generated data. Other approaches such as divergence
minimisation and moment matching have also been explored in the GAN literature,
and we synthesise these views to form an understanding in terms of the
relationships between them and the wider literature, highlighting avenues for
future exploration and cross-pollination.
Deceiving Google's Perspective API Built for Detecting Toxic Comments
Authors: Hossein Hosseini, Sreeram Kannan, Baosen Zhang, Radha Poovendran
Date: 2017-02-27 00:00:00

Social media platforms provide an environment where people can freely engage
in discussions. Unfortunately, they also enable several problems, such as
online harassment. Recently, Google and Jigsaw started a project called
Perspective, which uses machine learning to automatically detect toxic
language. A demonstration website has been also launched, which allows anyone
to type a phrase in the interface and instantaneously see the toxicity score
[1]. In this paper, we propose an attack on the Perspective toxic detection
system based on the adversarial examples. We show that an adversary can subtly
modify a highly toxic phrase in a way that the system assigns significantly
lower toxicity score to it. We apply the attack on the sample phrases provided
in the Perspective website and show that we can consistently reduce the
toxicity scores to the level of the non-toxic phrases. The existence of such
adversarial examples is very harmful for toxic detection systems and seriously
undermines their usability.
Stochastic Patching Process
Authors: Xuhui Fan, Bin Li, Yi Wang, Yang Wang, Fang Chen
Date: 2016-05-23 00:00:00

Stochastic partition models tailor a product space into a number of
rectangular regions such that the data within each region exhibit certain types
of homogeneity. Due to constraints of partition strategy, existing models may
cause unnecessary dissections in sparse regions when fitting data in dense
regions. To alleviate this limitation, we propose a parsimonious partition
model, named Stochastic Patching Process (SPP), to deal with multi-dimensional
arrays. SPP adopts an "enclosing" strategy to attach rectangular patches to
dense regions. SPP is self-consistent such that it can be extended to infinite
arrays. We apply SPP to relational modeling and the experimental results
validate its merit compared to the state-of-the-arts.
TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
Authors: Adji B. Dieng, Chong Wang, Jianfeng Gao, John Paisley
Date: 2016-11-05 00:00:00
Tasks: Language Modeling, Language Modelling, Sentiment Analysis, Topic Models

In this paper, we propose TopicRNN, a recurrent neural network (RNN)-based
language model designed to directly capture the global semantic meaning
relating words in a document via latent topics. Because of their sequential
nature, RNNs are good at capturing the local structure of a word sequence -
both semantic and syntactic - but might face difficulty remembering long-range
dependencies. Intuitively, these long-range dependencies are of semantic
nature. In contrast, latent topic models are able to capture the global
underlying semantic structure of a document but do not account for word
ordering. The proposed TopicRNN model integrates the merits of RNNs and latent
topic models: it captures local (syntactic) dependencies using an RNN and
global (semantic) dependencies using latent topics. Unlike previous work on
contextual RNN language modeling, our model is learned end-to-end. Empirical
results on word prediction show that TopicRNN outperforms existing contextual
RNN baselines. In addition, TopicRNN can be used as an unsupervised feature
extractor for documents. We do this for sentiment analysis on the IMDB movie
review dataset and report an error rate of $6.28\%$. This is comparable to the
state-of-the-art $5.91\%$ resulting from a semi-supervised approach. Finally,
TopicRNN also yields sensible topics, making it a useful alternative to
document models such as latent Dirichlet allocation.
Analyzing Linear Dynamical Systems: From Modeling to Coding and Learning
Authors: Wenbing Huang, Fuchun Sun, Lele Cao, Mehrtash Harandi
Date: 2016-08-03 00:00:00
Tasks: Dictionary Learning, General Classification, Time Series, Time Series Analysis, Video Classification, Video Segmentation, Video Semantic Segmentation

Encoding time-series with Linear Dynamical Systems (LDSs) leads to rich
models with applications ranging from dynamical texture recognition to video
segmentation to name a few. In this paper, we propose to represent LDSs with
infinite-dimensional subspaces and derive an analytic solution to obtain stable
LDSs. We then devise efficient algorithms to perform sparse coding and
dictionary learning on the space of infinite-dimensional subspaces. In
particular, two solutions are developed to sparsely encode an LDS. In the first
method, we map the subspaces into a Reproducing Kernel Hilbert Space (RKHS) and
achieve our goal through kernel sparse coding. As for the second solution, we
propose to embed the infinite-dimensional subspaces into the space of symmetric
matrices and formulate the sparse coding accordingly in the induced space. For
dictionary learning, we encode time-series by introducing a novel concept,
namely the two-fold LDSs. We then make use of the two-fold LDSs to derive an
analytical form for updating atoms of an LDS dictionary, i.e., each atom is an
LDS itself. Compared to several baselines and state-of-the-art methods, the
proposed methods yield higher accuracies in various classification tasks
including video classification and tactile recognition.
PLATO: Policy Learning using Adaptive Trajectory Optimization
Authors: Gregory Kahn, Tianhao Zhang, Sergey Levine, Pieter Abbeel
Date: 2016-03-02 00:00:00
Tasks: Model Predictive Control

Policy search can in principle acquire complex strategies for control of
robots and other autonomous systems. When the policy is trained to process raw
sensory inputs, such as images and depth maps, it can also acquire a strategy
that combines perception and control. However, effectively processing such
complex inputs requires an expressive policy class, such as a large neural
network. These high-dimensional policies are difficult to train, especially
when learning to control safety-critical systems. We propose PLATO, an
algorithm that trains complex control policies with supervised learning, using
model-predictive control (MPC) to generate the supervision, hence never in need
of running a partially trained and potentially unsafe policy. PLATO uses an
adaptive training method to modify the behavior of MPC to gradually match the
learned policy in order to generate training samples at states that are likely
to be visited by the learned policy. PLATO also maintains the MPC cost as an
objective to avoid highly undesirable actions that would result from strictly
following the learned policy before it has been fully trained. We prove that
this type of adaptive MPC expert produces supervision that leads to good
long-horizon performance of the resulting policy. We also empirically
demonstrate that MPC can still avoid dangerous on-policy actions in unexpected
situations during training. Our empirical results on a set of challenging
simulated aerial vehicle tasks demonstrate that, compared to prior methods,
PLATO learns faster, experiences substantially fewer catastrophic failures
(crashes) during training, and often converges to a better policy.
Memory Visualization for Gated Recurrent Neural Networks in Speech Recognition
Authors: Zhiyuan Tang, Ying Shi, Dong Wang, Yang Feng, Shiyue Zhang
Date: 2016-09-28 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Recurrent neural networks (RNNs) have shown clear superiority in sequence
modeling, particularly the ones with gated units, such as long short-term
memory (LSTM) and gated recurrent unit (GRU). However, the dynamic properties
behind the remarkable performance remain unclear in many applications, e.g.,
automatic speech recognition (ASR). This paper employs visualization techniques
to study the behavior of LSTM and GRU when performing speech recognition tasks.
Our experiments show some interesting patterns in the gated memory, and some of
them have inspired simple yet effective modifications on the network structure.
We report two of such modifications: (1) lazy cell update in LSTM, and (2)
shortcut connections for residual learning. Both modifications lead to more
comprehensible and powerful networks.