pwc-archive/papers-with-abstracts huggingface dataset:
Snorkel: Rapid Training Data Creation with Weak Supervision
Authors: Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, Christopher Ré
Date: 2017-11-28 00:00:00
Tasks: BIG-bench Machine Learning

Labeling training data is increasingly the largest bottleneck in deploying
machine learning systems. We present Snorkel, a first-of-its-kind system that
enables users to train state-of-the-art models without hand labeling any
training data. Instead, users write labeling functions that express arbitrary
heuristics, which can have unknown accuracies and correlations. Snorkel
denoises their outputs without access to ground truth by incorporating the
first end-to-end implementation of our recently proposed machine learning
paradigm, data programming. We present a flexible interface layer for writing
labeling functions based on our experience over the past year collaborating
with companies, agencies, and research labs. In a user study, subject matter
experts build models 2.8x faster and increase predictive performance an average
45.5% versus seven hours of hand labeling. We study the modeling tradeoffs in
this new setting and propose an optimizer for automating tradeoff decisions
that gives up to 1.8x speedup per pipeline execution. In two collaborations,
with the U.S. Department of Veterans Affairs and the U.S. Food and Drug
Administration, and on four open-source text and image data sets representative
of other deployments, Snorkel provides 132% average improvements to predictive
performance over prior heuristic approaches and comes within an average 3.60%
of the predictive performance of large hand-curated training sets.
Do Neural Nets Learn Statistical Laws behind Natural Language?
Authors: Shuntaro Takahashi, Kumiko Tanaka-Ishii
Date: 2017-07-16 00:00:00
Tasks: Deep Learning, Language Modeling, Language Modelling

The performance of deep learning in natural language processing has been
spectacular, but the reasons for this success remain unclear because of the
inherent complexity of deep learning. This paper provides empirical evidence of
its effectiveness and of a limitation of neural networks for language
engineering. Precisely, we demonstrate that a neural language model based on
long short-term memory (LSTM) effectively reproduces Zipf's law and Heaps' law,
two representative statistical properties underlying natural language. We
discuss the quality of reproducibility and the emergence of Zipf's law and
Heaps' law as training progresses. We also point out that the neural language
model has a limitation in reproducing long-range correlation, another
statistical property of natural language. This understanding could provide a
direction for improving the architectures of neural networks.
Deformation estimation of an elastic object by partial observation using a neural network
Authors: Utako Yamamoto, Megumi Nakao, Masayuki Ohzeki, Tetsuya Matsuda
Date: 2017-11-28 00:00:00
Tasks: Object

Deformation estimation of elastic object assuming an internal organ is
important for the computer navigation of surgery. The aim of this study is to
estimate the deformation of an entire three-dimensional elastic object using
displacement information of very few observation points. A learning approach
with a neural network was introduced to estimate the entire deformation of an
object. We applied our method to two elastic objects; a rectangular
parallelepiped model, and a human liver model reconstructed from computed
tomography data. The average estimation error for the human liver model was
0.041 mm when the object was deformed up to 66.4 mm, from only around 3 %
observations. These results indicate that the deformation of an entire elastic
object can be estimated with an acceptable level of error from limited
observations by applying a trained neural network to a new deformation.
Restricting Greed in Training of Generative Adversarial Network
Authors: Haoxuan You, Zhicheng Jiao, Haojun Xu, Jie Li, Ying Wang, Xinbo Gao
Date: 2017-11-28 00:00:00
Tasks: Diversity, Generative Adversarial Network

Generative adversarial network (GAN) has gotten wide re-search interest in
the field of deep learning. Variations of GAN have achieved competitive results
on specific tasks. However, the stability of training and diversity of
generated instances are still worth studying further. Training of GAN can be
thought of as a greedy procedure, in which the generative net tries to make the
locally optimal choice (minimizing loss function of discriminator) in each
iteration. Unfortunately, this often makes generated data resemble only a few
modes of real data and rotate between modes. To alleviate these problems, we
propose a novel training strategy to restrict greed in training of GAN. With
help of our method, the generated samples can cover more instance modes with
more stable training process. Evaluating our method on several representative
datasets, we demonstrate superiority of improved training strategy on typical
GAN models with different distance metrics.
Revisiting hand-crafted feature for action recognition: a set of improved dense trajectories
Authors: Kenji Matsui, Toru Tamaki, Gwladys Auffret, Bisser Raytchev, Kazufumi Kaneda
Date: 2017-11-28 00:00:00
Tasks: Action Recognition, Temporal Action Localization

We propose a feature for action recognition called Trajectory-Set (TS), on
top of the improved Dense Trajectory (iDT). The TS feature encodes only
trajectories around densely sampled interest points, without any appearance
features. Experimental results on the UCF50, UCF101, and HMDB51 action datasets
demonstrate that TS is comparable to state-of-the-arts, and outperforms many
other methods; for HMDB the accuracy of 85.4%, compared to the best accuracy of
80.2% obtained by a deep method. Our code is available on-line at
https://github.com/Gauffret/TrajectorySet .
Acoustic-To-Word Model Without OOV
Authors: Jinyu Li, Guoli Ye, Rui Zhao, Jasha Droppo, Yifan Gong
Date: 2017-11-28 00:00:00
Tasks: model

Recently, the acoustic-to-word model based on the Connectionist Temporal
Classification (CTC) criterion was shown as a natural end-to-end model directly
targeting words as output units. However, this type of word-based CTC model
suffers from the out-of-vocabulary (OOV) issue as it can only model limited
number of words in the output layer and maps all the remaining words into an
OOV output node. Therefore, such word-based CTC model can only recognize the
frequent words modeled by the network output nodes. It also cannot easily
handle the hot-words which emerge after the model is trained. In this study, we
improve the acoustic-to-word model with a hybrid CTC model which can predict
both words and characters at the same time. With a shared-hidden-layer
structure and modular design, the alignments of words generated from the
word-based CTC and the character-based CTC are synchronized. Whenever the
acoustic-to-word model emits an OOV token, we back off that OOV segment to the
word output generated from the character-based CTC, hence solving the OOV or
hot-words issue. Evaluated on a Microsoft Cortana voice assistant task, the
proposed model can reduce the errors introduced by the OOV output token in the
acoustic-to-word model by 30%.
Unsupervised Discovery of Structured Acoustic Tokens with Applications to Spoken Term Detection
Authors: Cheng-Tao Chung, Lin-shan Lee
Date: 2017-11-28 00:00:00

In this paper, we compare two paradigms for unsupervised discovery of
structured acoustic tokens directly from speech corpora without any human
annotation. The Multigranular Paradigm seeks to capture all available
information in the corpora with multiple sets of tokens for different model
granularities. The Hierarchical Paradigm attempts to jointly learn several
levels of signal representations in a hierarchical structure. The two paradigms
are unified within a theoretical framework in this paper. Query-by-Example
Spoken Term Detection (QbE-STD) experiments on the QUESST dataset of MediaEval
2015 verifies the competitiveness of the acoustic tokens. The Enhanced
Relevance Score (ERS) proposed in this work improves both paradigms for the
task of QbE-STD. We also list results on the ABX evaluation task of the Zero
Resource Challenge 2015 for comparison of the Paradigms.
A fatal point concept and a low-sensitivity quantitative measure for traffic safety analytics
Authors: Shan Suthaharan
Date: 2017-11-28 00:00:00
Tasks: Clustering, Sensitivity

The variability of the clusters generated by clustering techniques in the
domain of latitude and longitude variables of fatal crash data are
significantly unpredictable. This unpredictability, caused by the randomness of
fatal crash incidents, reduces the accuracy of crash frequency (i.e., counts of
fatal crashes per cluster) which is used to measure traffic safety in practice.
In this paper, a quantitative measure of traffic safety that is not
significantly affected by the aforementioned variability is proposed. It
introduces a fatal point -- a segment with the highest frequency of fatality --
concept based on cluster characteristics and detects them by imposing rounding
errors to the hundredth decimal place of the longitude. The frequencies of the
cluster and the cluster's fatal point are combined to construct a low-sensitive
quantitative measure of traffic safety for the cluster. The performance of the
proposed measure of traffic safety is then studied by varying the parameter k
of k-means clustering with the expectation that other clustering techniques can
be adopted in a similar fashion. The 2015 North Carolina fatal crash dataset of
Fatality Analysis Reporting System (FARS) is used to evaluate the proposed
fatal point concept and perform experimental analysis to determine the
effectiveness of the proposed measure. The empirical study shows that the
average traffic safety, measured by the proposed quantitative measure over
several clusters, is not significantly affected by the variability, compared to
that of the standard crash frequency.
Saliency Prediction for Mobile User Interfaces
Authors: Prakhar Gupta, Shubh Gupta, Ajaykrishnan Jayagopal, Sourav Pal, Ritwik Sinha
Date: 2017-11-10 00:00:00
Tasks: Prediction, Saliency Prediction

We introduce models for saliency prediction for mobile user interfaces. A
mobile interface may include elements like buttons, text, etc. in addition to
natural images which enable performing a variety of tasks. Saliency in natural
images is a well studied area. However, given the difference in what
constitutes a mobile interface, and the usage context of these devices, we
postulate that saliency prediction for mobile interface images requires a fresh
approach. Mobile interface design involves operating on elements, the building
blocks of the interface. We first collected eye-gaze data from mobile devices
for free viewing task. Using this data, we develop a novel autoencoder based
multi-scale deep learning model that provides saliency prediction at the mobile
interface element level. Compared to saliency prediction approaches developed
for natural images, we show that our approach performs significantly better on
a range of established metrics.
Differential Performance Debugging with Discriminant Regression Trees
Authors: Saeid Tizpaz-Niari, Pavol Cerny, Bor-Yuh Evan Chang, Ashutosh Trivedi
Date: 2017-11-11 00:00:00
Tasks: Clustering, regression

Differential performance debugging is a technique to find performance
problems. It applies in situations where the performance of a program is
(unexpectedly) different for different classes of inputs. The task is to
explain the differences in asymptotic performance among various input classes
in terms of program internals. We propose a data-driven technique based on
discriminant regression tree (DRT) learning problem where the goal is to
discriminate among different classes of inputs. We propose a new algorithm for
DRT learning that first clusters the data into functional clusters, capturing
different asymptotic performance classes, and then invokes off-the-shelf
decision tree learning algorithms to explain these clusters. We focus on linear
functional clusters and adapt classical clustering algorithms (K-means and
spectral) to produce them. For the K-means algorithm, we generalize the notion
of the cluster centroid from a point to a linear function. We adapt spectral
clustering by defining a novel kernel function to capture the notion of linear
similarity between two data points. We evaluate our approach on benchmarks
consisting of Java programs where we are interested in debugging performance.
We show that our algorithm significantly outperforms other well-known
regression tree learning algorithms in terms of running time and accuracy of
classification.
Vietnamese Semantic Role Labelling
Authors: Phuong Le-Hong, Thai Hoang Pham, Xuan Khoai Pham, Thi Minh Huyen Nguyen, Thi Luong Nguyen, Minh Hiep Nguyen
Date: 2017-11-28 00:00:00
Tasks: Semantic Parsing

In this paper, we study semantic role labelling (SRL), a subtask of semantic
parsing of natural language sentences and its application for the Vietnamese
language. We present our effort in building Vietnamese PropBank, the first
Vietnamese SRL corpus and a software system for labelling semantic roles of
Vietnamese texts. In particular, we present a novel constituent extraction
algorithm in the argument candidate identification step which is more suitable
and more accurate than the common node-mapping method. In the machine learning
part, our system integrates distributed word features produced by two recent
unsupervised learning models in two learned statistical classifiers and makes
use of integer linear programming inference procedure to improve the accuracy.
The system is evaluated in a series of experiments and achieves a good result,
an $F_1$ score of 74.77%. Our system, including corpus and software, is
available as an open source project for free research and we believe that it is
a good baseline for the development of future Vietnamese SRL systems.
Homomorphic Parameter Compression for Distributed Deep Learning Training
Authors: Jaehee Jang, Byungook Na, Sungroh Yoon
Date: 2017-11-28 00:00:00
Tasks: Deep Learning

Distributed training of deep neural networks has received significant
research interest, and its major approaches include implementations on multiple
GPUs and clusters. Parallelization can dramatically improve the efficiency of
training deep and complicated models with large-scale data. A fundamental
barrier against the speedup of DNN training, however, is the trade-off between
computation and communication time. In other words, increasing the number of
worker nodes decreases the time consumed in computation while simultaneously
increasing communication overhead under constrained network bandwidth,
especially in commodity hardware environments. To alleviate this trade-off, we
suggest the idea of homomorphic parameter compression, which compresses
parameters with the least expense and trains the DNN with the compressed
representation. Although the specific method is yet to be discovered, we
demonstrate that there is a high probability that the homomorphism can reduce
the communication overhead, thanks to little compression and decompression
times. We also provide theoretical speedup of homomorphic compression.
Automatic Streaming Segmentation of Stereo Video Using Bilateral Space
Authors: Wenjing Ke, Yuanjie Zhu, Lei Yu
Date: 2017-10-10 00:00:00

In this paper, we take advantage of binocular camera and propose an
unsupervised algorithm based on semi-supervised segmentation algorithm and
extracting foreground part efficiently. We creatively embed depth information
into bilateral grid in the graph cut model and achieve considerable segmenting
accuracy in the case of no user input. The experi- ment approves the high
precision, time efficiency of our algorithm and its adaptation to complex
natural scenario which is significant for practical application.
Hybrid Reward Architecture for Reinforcement Learning
Authors: Harm van Seijen, Mehdi Fatemi, Joshua Romoff, Romain Laroche, Tavian Barnes, Jeffrey Tsang
Date: 2017-06-13 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Representation Learning

One of the main challenges in reinforcement learning (RL) is generalisation.
In typical deep RL methods this is achieved by approximating the optimal value
function with a low-dimensional representation using a deep network. While this
approach works well in many domains, in domains where the optimal value
function cannot easily be reduced to a low-dimensional representation, learning
can be very slow and unstable. This paper contributes towards tackling such
challenging domains, by proposing a new method, called Hybrid Reward
Architecture (HRA). HRA takes as input a decomposed reward function and learns
a separate value function for each component reward function. Because each
component typically only depends on a subset of all features, the corresponding
value function can be approximated more easily by a low-dimensional
representation, enabling more effective learning. We demonstrate HRA on a
toy-problem and the Atari game Ms. Pac-Man, where HRA achieves above-human
performance.
A Parameter-Free Learning Automaton Scheme
Authors: Hao Ge
Date: 2017-11-28 00:00:00
Tasks: Bayesian Inference

For a learning automaton, a proper configuration of its learning parameters,
which are crucial for the automaton's performance, is relatively difficult due
to the necessity of a manual parameter tuning before real applications. To
ensure a stable and reliable performance in stochastic environments, parameter
tuning can be a time-consuming and interaction-costing procedure in the field
of LA. Especially, it is a fatal limitation for LA-based applications where the
interactions with environments are expensive.
  In this paper, we propose a parameter-free learning automaton scheme to avoid
parameter tuning by a Bayesian inference method. In contrast to existing
schemes where the parameters should be carefully tuned according to the
environment, the performance of this scheme is not sensitive to external
environments because a set of parameters can be consistently applied to various
environments, which dramatically reduce the difficulty of applying a learning
automaton to an unknown stochastic environment. A rigorous proof of
$\epsilon$-optimality for the proposed scheme is provided and numeric
experiments are carried out on benchmark environments to verify its
effectiveness. The results show that, without any parameter tuning cost, the
proposed parameter-free learning automaton (PFLA) can achieve a competitive
performance compared with other well-tuned schemes and outperform untuned
schemes on consistency of performance.
3D-A-Nets: 3D Deep Dense Descriptor for Volumetric Shapes with Adversarial Networks
Authors: Mengwei Ren, Liang Niu, Yi Fang
Date: 2017-11-28 00:00:00
Tasks: 3D Shape Classification, Clustering, Retrieval

Recently researchers have been shifting their focus towards learned 3D shape
descriptors from hand-craft ones to better address challenging issues of the
deformation and structural variation inherently present in 3D objects. 3D
geometric data are often transformed to 3D Voxel grids with regular format in
order to be better fed to a deep neural net architecture. However, the
computational intractability of direct application of 3D convolutional nets to
3D volumetric data severely limits the efficiency (i.e. slow processing) and
effectiveness (i.e. unsatisfied accuracy) in processing 3D geometric data. In
this paper, powered with a novel design of adversarial networks (3D-A-Nets), we
have developed a novel 3D deep dense shape descriptor (3D-DDSD) to address the
challenging issues of efficient and effective 3D volumetric data processing. We
developed new definition of 2D multilayer dense representation (MDR) of 3D
volumetric data to extract concise but geometrically informative shape
description and a novel design of adversarial networks that jointly train a set
of convolution neural network (CNN), recurrent neural network (RNN) and an
adversarial discriminator. More specifically, the generator network produces 3D
shape features that encourages the clustering of samples from the same category
with correct class label, whereas the discriminator network discourages the
clustering by assigning them misleading adversarial class labels. By addressing
the challenges posed by the computational inefficiency of direct application of
CNN to 3D volumetric data, 3D-A-Nets can learn high-quality 3D-DSDD which
demonstrates superior performance on 3D shape classification and retrieval over
other state-of-the-art techniques by a great margin.
Learning Channel Inter-dependencies at Multiple Scales on Dense Networks for Face Recognition
Authors: Qiangchang Wang, Guodong Guo, Mohammad Iqbal Nouyed
Date: 2017-11-28 00:00:00
Tasks: Face Recognition

We propose a new deep network structure for unconstrained face recognition.
The proposed network integrates several key components together in order to
characterize complex data distributions, such as in unconstrained face images.
Inspired by recent progress in deep networks, we consider some important
concepts, including multi-scale feature learning, dense connections of network
layers, and weighting different network flows, for building our deep network
structure. The developed network is evaluated in unconstrained face matching,
showing the capability of learning complex data distributions caused by face
images with various qualities.
An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks
Authors: Qinglong Wang, Kaixuan Zhang, Alexander G. Ororbia II, Xinyu Xing, Xue Liu, C. Lee Giles
Date: 2017-09-29 00:00:00
Tasks: Medical Diagnosis

Rule extraction from black-box models is critical in domains that require
model validation before implementation, as can be the case in credit scoring
and medical diagnosis. Though already a challenging problem in statistical
learning in general, the difficulty is even greater when highly non-linear,
recursive models, such as recurrent neural networks (RNNs), are fit to data.
Here, we study the extraction of rules from second-order recurrent neural
networks trained to recognize the Tomita grammars. We show that production
rules can be stably extracted from trained RNNs and that in certain cases the
rules outperform the trained RNNs.