pwc-archive/papers-with-abstracts huggingface dataset:
A Generalised Directional Laplacian Distribution: Estimation, Mixture Models and Audio Source Separation
Authors: Nikolaos Mitianoudis
Date: 2017-08-16 00:00:00
Tasks: Audio Source Separation

Directional or Circular statistics are pertaining to the analysis and
interpretation of directions or rotations. In this work, a novel probability
distribution is proposed to model multidimensional sparse directional data. The
Generalised Directional Laplacian Distribution (DLD) is a hybrid between the
Laplacian distribution and the von Mises-Fisher distribution. The
distribution's parameters are estimated using Maximum-Likelihood Estimation
over a set of training data points. Mixtures of Directional Laplacian
Distributions (MDLD) are also introduced in order to model multiple
concentrations of sparse directional data. The author explores the application
of the derived DLD mixture model to cluster sound sources that exist in an
underdetermined instantaneous sound mixture. The proposed model can solve the
general K x L (K<L) underdetermined instantaneous source separation problem,
offering a fast and stable solution.
Language Identification Using Deep Convolutional Recurrent Neural Networks
Authors: Christian Bartz, Tom Herold, Haojin Yang, Christoph Meinel
Date: 2017-08-16 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), General Classification, Language Identification, speech-recognition, Speech Recognition, Spoken language identification

Language Identification (LID) systems are used to classify the spoken
language from a given audio sample and are typically the first step for many
spoken language processing tasks, such as Automatic Speech Recognition (ASR)
systems. Without automatic language detection, speech utterances cannot be
parsed correctly and grammar rules cannot be applied, causing subsequent speech
recognition steps to fail. We propose a LID system that solves the problem in
the image domain, rather than the audio domain. We use a hybrid Convolutional
Recurrent Neural Network (CRNN) that operates on spectrogram images of the
provided audio snippets. In extensive experiments we show, that our model is
applicable to a range of noisy scenarios and can easily be extended to
previously unknown languages, while maintaining its classification accuracy. We
release our code and a large scale training set for LID systems to the
community.
Efficiently Tracking Homogeneous Regions in Multichannel Images
Authors: Tobias Böttger, Christina Eisenhofer
Date: 2017-08-16 00:00:00
Tasks: Object Tracking

We present a method for tracking Maximally Stable Homogeneous Regions (MSHR)
in images with an arbitrary number of channels. MSHR are conceptionally very
similar to Maximally Stable Extremal Regions (MSER) and Maximally Stable Color
Regions (MSCR), but can also be applied to hyperspectral and color images while
remaining extremely efficient. The presented approach makes use of the
edge-based component-tree which can be calculated in linear time. In the
tracking step, the MSHR are localized by matching them to the nodes in the
component-tree. We use rotationally invariant region and gray-value features
that can be calculated through first and second order moments at low
computational complexity. Furthermore, we use a weighted feature vector to
improve the data association in the tracking step. The algorithm is evaluated
on a collection of different tracking scenes from the literature. Furthermore,
we present two different applications: 2D object tracking and the 3D
segmentation of organs.
Weighted parallel SGD for distributed unbalanced-workload training system
Authors: Cheng Daning, Li Shigang, Zhang Yunquan
Date: 2017-08-16 00:00:00
Tasks: Stochastic Optimization

Stochastic gradient descent (SGD) is a popular stochastic optimization method
in machine learning. Traditional parallel SGD algorithms, e.g., SimuParallel
SGD, often require all nodes to have the same performance or to consume equal
quantities of data. However, these requirements are difficult to satisfy when
the parallel SGD algorithms run in a heterogeneous computing environment;
low-performance nodes will exert a negative influence on the final result. In
this paper, we propose an algorithm called weighted parallel SGD (WP-SGD).
WP-SGD combines weighted model parameters from different nodes in the system to
produce the final output. WP-SGD makes use of the reduction in standard
deviation to compensate for the loss from the inconsistency in performance of
nodes in the cluster, which means that WP-SGD does not require that all nodes
consume equal quantities of data. We also analyze the theoretical feasibility
of running two other parallel SGD algorithms combined with WP-SGD in a
heterogeneous environment. The experimental results show that WP-SGD
significantly outperforms the traditional parallel SGD algorithms on
distributed training systems with an unbalanced workload.
Brain Responses During Robot-Error Observation
Authors: Dominik Welke, Joos Behncke, Marina Hader, Robin Tibor Schirrmeister, Andreas Schönau, Boris Eßmann, Oliver Müller, Wolfram Burgard, Tonio Ball
Date: 2017-08-04 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

Brain-controlled robots are a promising new type of assistive device for
severely impaired persons. Little is however known about how to optimize the
interaction of humans and brain-controlled robots. Information about the
human's perceived correctness of robot performance might provide a useful
teaching signal for adaptive control algorithms and thus help enhancing robot
control. Here, we studied whether watching robots perform erroneous vs. correct
action elicits differential brain responses that can be decoded from single
trials of electroencephalographic (EEG) recordings, and whether brain activity
during human-robot interaction is modulated by the robot's visual similarity to
a human. To address these topics, we designed two experiments. In experiment I,
participants watched a robot arm pour liquid into a cup. The robot performed
the action either erroneously or correctly, i.e. it either spilled some liquid
or not. In experiment II, participants observed two different types of robots,
humanoid and non-humanoid, grabbing a ball. The robots either managed to grab
the ball or not. We recorded high-resolution EEG during the observation tasks
in both experiments to train a Filter Bank Common Spatial Pattern (FBCSP)
pipeline on the multivariate EEG signal and decode for the correctness of the
observed action, and for the type of the observed robot. Our findings show that
it was possible to decode both correctness and robot type for the majority of
participants significantly, although often just slightly, above chance level.
Our findings suggest that non-invasive recordings of brain responses elicited
when observing robots indeed contain decodable information about the
correctness of the robot's action and the type of observed robot.
Efficient Compression Technique for Sparse Sets
Authors: Rameshwar Pratap, Ishan Sohony, Raghav Kulkarni
Date: 2017-08-16 00:00:00
Tasks: Clustering

Recent technological advancements have led to the generation of huge amounts
of data over the web, such as text, image, audio and video. Most of this data
is high dimensional and sparse, for e.g., the bag-of-words representation used
for representing text. Often, an efficient search for similar data points needs
to be performed in many applications like clustering, nearest neighbour search,
ranking and indexing. Even though there have been significant increases in
computational power, a simple brute-force similarity-search on such datasets is
inefficient and at times impossible. Thus, it is desirable to get a compressed
representation which preserves the similarity between data points. In this
work, we consider the data points as sets and use Jaccard similarity as the
similarity measure. Compression techniques are generally evaluated on the
following parameters --1) Randomness required for compression, 2) Time required
for compression, 3) Dimension of the data after compression, and 4) Space
required to store the compressed data. Ideally, the compressed representation
of the data should be such, that the similarity between each pair of data
points is preserved, while keeping the time and the randomness required for
compression as low as possible.
  We show that the compression technique suggested by Pratap and Kulkarni also
works well for Jaccard similarity. We present a theoretical proof of the same
and complement it with rigorous experimentations on synthetic as well as
real-world datasets. We also compare our results with the state-of-the-art
"min-wise independent permutation", and show that our compression algorithm
achieves almost equal accuracy while significantly reducing the compression
time and the randomness.
BitNet: Bit-Regularized Deep Neural Networks
Authors: Aswin Raghavan, Mohamed Amer, Sek Chai, Graham Taylor
Date: 2017-08-16 00:00:00
Tasks: Translation

We present a novel optimization strategy for training neural networks which
we call "BitNet". The parameters of neural networks are usually unconstrained
and have a dynamic range dispersed over all real values. Our key idea is to
limit the expressive power of the network by dynamically controlling the range
and set of values that the parameters can take. We formulate this idea using a
novel end-to-end approach that circumvents the discrete parameter space by
optimizing a relaxed continuous and differentiable upper bound of the typical
classification loss function. The approach can be interpreted as a
regularization inspired by the Minimum Description Length (MDL) principle. For
each layer of the network, our approach optimizes real-valued translation and
scaling factors and arbitrary precision integer-valued parameters (weights). We
empirically compare BitNet to an equivalent unregularized model on the MNIST
and CIFAR-10 datasets. We show that BitNet converges faster to a superior
quality solution. Additionally, the resulting model has significant savings in
memory due to the use of integer-valued parameters.
Global Normalization of Convolutional Neural Networks for Joint Entity and Relation Classification
Authors: Heike Adel, Hinrich Schütze
Date: 2017-07-24 00:00:00
Proceeding: EMNLP 2017 9
Tasks: General Classification, Relation, Relation Classification, Relation Extraction

We introduce globally normalized convolutional neural networks for joint
entity classification and relation extraction. In particular, we propose a way
to utilize a linear-chain conditional random field output layer for predicting
entity types and relations between entities at the same time. Our experiments
show that global normalization outperforms a locally normalized softmax layer
on a benchmark dataset.
CNN-based Cascaded Multi-task Learning of High-level Prior and Density Estimation for Crowd Counting
Authors: Vishwanath A. Sindagi, Vishal M. Patel
Date: 2017-07-30 00:00:00
Tasks: Crowd Counting, Density Estimation, General Classification, Multi-Task Learning

Estimating crowd count in densely crowded scenes is an extremely challenging
task due to non-uniform scale variations. In this paper, we propose a novel
end-to-end cascaded network of CNNs to jointly learn crowd count classification
and density map estimation. Classifying crowd count into various groups is
tantamount to coarsely estimating the total count in the image thereby
incorporating a high-level prior into the density estimation network. This
enables the layers in the network to learn globally relevant discriminative
features which aid in estimating highly refined density maps with lower count
error. The joint training is performed in an end-to-end fashion. Extensive
experiments on highly challenging publicly available datasets show that the
proposed method achieves lower count error and better quality density maps as
compared to the recent state-of-the-art methods.
StarCraft II: A New Challenge for Reinforcement Learning
Authors: Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John Agapiou, Julian Schrittwieser, John Quan, Stephen Gaffney, Stig Petersen, Karen Simonyan, Tom Schaul, Hado van Hasselt, David Silver, Timothy Lillicrap, Kevin Calderone, Paul Keet, Anthony Brunasso, David Lawrence, Anders Ekermo, Jacob Repp, Rodney Tsing
Date: 2017-08-16 00:00:00
Tasks: Deep Reinforcement Learning, Real-Time Strategy Games, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Starcraft, Starcraft II

This paper introduces SC2LE (StarCraft II Learning Environment), a
reinforcement learning environment based on the StarCraft II game. This domain
poses a new grand challenge for reinforcement learning, representing a more
difficult class of problems than considered in most prior work. It is a
multi-agent problem with multiple players interacting; there is imperfect
information due to a partially observed map; it has a large action space
involving the selection and control of hundreds of units; it has a large state
space that must be observed solely from raw input feature planes; and it has
delayed credit assignment requiring long-term strategies over thousands of
steps. We describe the observation, action, and reward specification for the
StarCraft II domain and provide an open source Python-based interface for
communicating with the game engine. In addition to the main game maps, we
provide a suite of mini-games focusing on different elements of StarCraft II
gameplay. For the main game maps, we also provide an accompanying dataset of
game replay data from human expert players. We give initial baseline results
for neural networks trained from this data to predict game outcomes and player
actions. Finally, we present initial baseline results for canonical deep
reinforcement learning agents applied to the StarCraft II domain. On the
mini-games, these agents learn to achieve a level of play that is comparable to
a novice player. However, when trained on the main game, these agents are
unable to make significant progress. Thus, SC2LE offers a new and challenging
environment for exploring deep reinforcement learning algorithms and
architectures.
Racing Thompson: an Efficient Algorithm for Thompson Sampling with Non-conjugate Priors
Authors: Yichi Zhou, Jun Zhu, Jingwei Zhuo
Date: 2017-08-16 00:00:00
Proceeding: ICML 2018 7
Tasks: Thompson Sampling

Thompson sampling has impressive empirical performance for many multi-armed
bandit problems. But current algorithms for Thompson sampling only work for the
case of conjugate priors since these algorithms require to infer the posterior,
which is often computationally intractable when the prior is not conjugate. In
this paper, we propose a novel algorithm for Thompson sampling which only
requires to draw samples from a tractable distribution, so our algorithm is
efficient even when the prior is non-conjugate. To do this, we reformulate
Thompson sampling as an optimization problem via the Gumbel-Max trick. After
that we construct a set of random variables and our goal is to identify the one
with highest mean. Finally, we solve it with techniques in best arm
identification.
Modality-specific Cross-modal Similarity Measurement with Recurrent Attention Network
Authors: Yuxin Peng, Jinwei Qi, Yuxin Yuan
Date: 2017-08-16 00:00:00
Tasks: Cross-Modal Retrieval, Retrieval, Sentence

Nowadays, cross-modal retrieval plays an indispensable role to flexibly find
information across different modalities of data. Effectively measuring the
similarity between different modalities of data is the key of cross-modal
retrieval. Different modalities such as image and text have imbalanced and
complementary relationships, which contain unequal amount of information when
describing the same semantics. For example, images often contain more details
that cannot be demonstrated by textual descriptions and vice versa. Existing
works based on Deep Neural Network (DNN) mostly construct one common space for
different modalities to find the latent alignments between them, which lose
their exclusive modality-specific characteristics. Different from the existing
works, we propose modality-specific cross-modal similarity measurement (MCSM)
approach by constructing independent semantic space for each modality, which
adopts end-to-end framework to directly generate modality-specific cross-modal
similarity without explicit common representation. For each semantic space,
modality-specific characteristics within one modality are fully exploited by
recurrent attention network, while the data of another modality is projected
into this space with attention based joint embedding to utilize the learned
attention weights for guiding the fine-grained cross-modal correlation
learning, which can capture the imbalanced and complementary relationships
between different modalities. Finally, the complementarity between the semantic
spaces for different modalities is explored by adaptive fusion of the
modality-specific cross-modal similarities to perform cross-modal retrieval.
Experiments on the widely-used Wikipedia and Pascal Sentence datasets as well
as our constructed large-scale XMediaNet dataset verify the effectiveness of
our proposed approach, outperforming 9 state-of-the-art methods.
Dialogue Act Segmentation for Vietnamese Human-Human Conversational Texts
Authors: Thi Lan Ngo, Khac Linh Pham, Minh Son Cao, Son Bao Pham, Xuan Hieu Phan
Date: 2017-08-16 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), BIG-bench Machine Learning, Deep Learning, Machine Translation, speech-recognition, Speech Recognition, Translation

Dialog act identification plays an important role in understanding
conversations. It has been widely applied in many fields such as dialogue
systems, automatic machine translation, automatic speech recognition, and
especially useful in systems with human-computer natural language dialogue
interfaces such as virtual assistants and chatbots. The first step of
identifying dialog act is identifying the boundary of the dialog act in
utterances. In this paper, we focus on segmenting the utterance according to
the dialog act boundaries, i.e. functional segments identification, for
Vietnamese utterances. We investigate carefully functional segment
identification in two approaches: (1) machine learning approach using maximum
entropy (ME) and conditional random fields (CRFs); (2) deep learning approach
using bidirectional Long Short-Term Memory (LSTM) with a CRF layer
(Bi-LSTM-CRF) on two different conversational datasets: (1) Facebook messages
(Message data); (2) transcription from phone conversations (Phone data). To the
best of our knowledge, this is the first work that applies deep learning based
approach to dialog act segmentation. As the results show, deep learning
approach performs appreciably better as to compare with traditional machine
learning approaches. Moreover, it is also the first study that tackles dialog
act and functional segment identification for Vietnamese.
Active Orthogonal Matching Pursuit for Sparse Subspace Clustering
Authors: Yanxi Chen, Gen Li, Yuantao Gu
Date: 2017-08-16 00:00:00
Tasks: Clustering

Sparse Subspace Clustering (SSC) is a state-of-the-art method for clustering
high-dimensional data points lying in a union of low-dimensional subspaces.
However, while $\ell_1$ optimization-based SSC algorithms suffer from high
computational complexity, other variants of SSC, such as Orthogonal Matching
Pursuit-based SSC (OMP-SSC), lose clustering accuracy in pursuit of improving
time efficiency. In this letter, we propose a novel Active OMP-SSC, which
improves clustering accuracy of OMP-SSC by adaptively updating data points and
randomly dropping data points in the OMP process, while still enjoying the low
computational complexity of greedy pursuit algorithms. We provide heuristic
analysis of our approach, and explain how these two active steps achieve a
better tradeoff between connectivity and separation. Numerical results on both
synthetic data and real-world data validate our analyses and show the
advantages of the proposed active algorithm.
DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding
Authors: Yinda Zhang, Mingru Bai, Pushmeet Kohli, Shahram Izadi, Jianxiong Xiao
Date: 2016-03-16 00:00:00
Proceeding: ICCV 2017 10
Tasks: Object, Scene Understanding

While deep neural networks have led to human-level performance on computer
vision tasks, they have yet to demonstrate similar gains for holistic scene
understanding. In particular, 3D context has been shown to be an extremely
important cue for scene understanding - yet very little research has been done
on integrating context information with deep models. This paper presents an
approach to embed 3D context into the topology of a neural network trained to
perform holistic scene understanding. Given a depth image depicting a 3D scene,
our network aligns the observed scene with a predefined 3D scene template, and
then reasons about the existence and location of each object within the scene
template. In doing so, our model recognizes multiple objects in a single
forward pass of a 3D convolutional neural network, capturing both global scene
and local object information simultaneously. To create training data for this
3D network, we generate partly hallucinated depth images which are rendered by
replacing real objects with a repository of CAD models of the same object
category. Extensive experiments demonstrate the effectiveness of our algorithm
compared to the state-of-the-arts. Source code and data are available at
http://deepcontext.cs.princeton.edu.
Driving Style Analysis Using Primitive Driving Patterns With Bayesian Nonparametric Approaches
Authors: Wenshuo Wang, Junqiang Xi, Ding Zhao
Date: 2017-08-16 00:00:00
Tasks: Time Series Analysis

Analysis and recognition of driving styles are profoundly important to
intelligent transportation and vehicle calibration. This paper presents a novel
driving style analysis framework using the primitive driving patterns learned
from naturalistic driving data. In order to achieve this, first, a Bayesian
nonparametric learning method based on a hidden semi-Markov model (HSMM) is
introduced to extract primitive driving patterns from time series driving data
without prior knowledge of the number of these patterns. In the Bayesian
nonparametric approach, we utilize a hierarchical Dirichlet process (HDP)
instead of learning the unknown number of smooth dynamical modes of HSMM, thus
generating the primitive driving patterns. Each primitive pattern is clustered
and then labeled using behavioral semantics according to drivers' physical and
psychological perception thresholds. For each driver, 75 primitive driving
patterns in car-following scenarios are learned and semantically labeled. In
order to show the HDP-HSMM's utility to learn primitive driving patterns, other
two Bayesian nonparametric approaches, HDP-HMM and sticky HDP-HMM, are
compared. The naturalistic driving data of 18 drivers were collected from the
University of Michigan Safety Pilot Model Deployment (SPDM) database. The
individual driving styles are discussed according to distribution
characteristics of the learned primitive driving patterns and also the
difference in driving styles among drivers are evaluated using the
Kullback-Leibler divergence. The experiment results demonstrate that the
proposed primitive pattern-based method can allow one to semantically
understand driver behaviors and driving styles.