pwc-archive/papers-with-abstracts huggingface dataset:
Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling
Authors: Qi Meng, Wei Chen, Yue Wang, Zhi-Ming Ma, Tie-Yan Liu
Date: 2017-09-29 00:00:00
Tasks: BIG-bench Machine Learning

When using stochastic gradient descent to solve large-scale machine learning
problems, a common practice of data processing is to shuffle the training data,
partition the data across multiple machines if needed, and then perform several
epochs of training on the re-shuffled (either locally or globally) data. The
above procedure makes the instances used to compute the gradients no longer
independently sampled from the training data set. Then does the distributed SGD
method have desirable convergence properties in this practical situation? In
this paper, we give answers to this question. First, we give a mathematical
formulation for the practical data processing procedure in distributed machine
learning, which we call data partition with global/local shuffling. We observe
that global shuffling is equivalent to without-replacement sampling if the
shuffling operations are independent. We prove that SGD with global shuffling
has convergence guarantee in both convex and non-convex cases. An interesting
finding is that, the non-convex tasks like deep learning are more suitable to
apply shuffling comparing to the convex tasks. Second, we conduct the
convergence analysis for SGD with local shuffling. The convergence rate for
local shuffling is slower than that for global shuffling, since it will lose
some information if there's no communication between partitioned data. Finally,
we consider the situation when the permutation after shuffling is not uniformly
distributed (insufficient shuffling), and discuss the condition under which
this insufficiency will not influence the convergence rate. Our theoretical
results provide important insights to large-scale machine learning, especially
in the selection of data processing methods in order to achieve faster
convergence and good speedup. Our theoretical findings are verified by
extensive experiments on logistic regression and deep neural networks.
The BURCHAK corpus: a Challenge Data Set for Interactive Learning of Visually Grounded Word Meanings
Authors: Yanchao Yu, Arash Eshghi, Gregory Mills, Oliver Joseph Lemon
Date: 2017-09-29 00:00:00
Proceeding: WS 2017 4
Tasks: Attribute, Reinforcement Learning, Sentence

We motivate and describe a new freely available human-human dialogue dataset
for interactive learning of visually grounded word meanings through ostensive
definition by a tutor to a learner. The data has been collected using a novel,
character-by-character variant of the DiET chat tool (Healey et al., 2003;
Mills and Healey, submitted) with a novel task, where a Learner needs to learn
invented visual attribute words (such as " burchak " for square) from a tutor.
As such, the text-based interactions closely resemble face-to-face conversation
and thus contain many of the linguistic phenomena encountered in natural,
spontaneous dialogue. These include self-and other-correction, mid-sentence
continuations, interruptions, overlaps, fillers, and hedges. We also present a
generic n-gram framework for building user (i.e. tutor) simulations from this
type of incremental data, which is freely available to researchers. We show
that the simulations produce outputs that are similar to the original data
(e.g. 78% turn match similarity). Finally, we train and evaluate a
Reinforcement Learning dialogue control agent for learning visually grounded
word meanings, trained from the BURCHAK corpus. The learned policy shows
comparable performance to a rule-based system built previously.
Jointly Attentive Spatial-Temporal Pooling Networks for Video-based Person Re-Identification
Authors: Shuangjie Xu, Yu Cheng, Kang Gu, Yang Yang, Shiyu Chang, Pan Zhou
Date: 2017-08-03 00:00:00
Proceeding: ICCV 2017 10
Tasks: Person Re-Identification, Video-Based Person Re-Identification

Person Re-Identification (person re-id) is a crucial task as its applications
in visual surveillance and human-computer interaction. In this work, we present
a novel joint Spatial and Temporal Attention Pooling Network (ASTPN) for
video-based person re-identification, which enables the feature extractor to be
aware of the current input video sequences, in a way that interdependency from
the matching items can directly influence the computation of each other's
representation. Specifically, the spatial pooling layer is able to select
regions from each frame, while the attention temporal pooling performed can
select informative frames over the sequence, both pooling guided by the
information from distance matching. Experiments are conduced on the iLIDS-VID,
PRID-2011 and MARS datasets and the results demonstrate that this approach
outperforms existing state-of-art methods. We also analyze how the joint
pooling in both dimensions can boost the person re-id performance more
effectively than using either of them separately.
Training an adaptive dialogue policy for interactive learning of visually grounded word meanings
Authors: Yanchao Yu, Arash Eshghi, Oliver Lemon
Date: 2017-09-29 00:00:00
Proceeding: WS 2016 9
Tasks: Semantic Parsing

We present a multi-modal dialogue system for interactive learning of
perceptually grounded word meanings from a human tutor. The system integrates
an incremental, semantic parsing/generation framework - Dynamic Syntax and Type
Theory with Records (DS-TTR) - with a set of visual classifiers that are
learned throughout the interaction and which ground the meaning representations
that it produces. We use this system in interaction with a simulated human
tutor to study the effects of different dialogue policies and capabilities on
the accuracy of learned meanings, learning rates, and efforts/costs to the
tutor. We show that the overall performance of the learning agent is affected
by (1) who takes initiative in the dialogues; (2) the ability to express/use
their confidence level about visual attributes; and (3) the ability to process
elliptical and incrementally constructed dialogue turns. Ultimately, we train
an adaptive dialogue policy which optimises the trade-off between classifier
accuracy and tutoring costs.
Learning how to learn: an adaptive dialogue agent for incrementally learning visually grounded word meanings
Authors: Yanchao Yu, Arash Eshghi, Oliver Lemon
Date: 2017-09-29 00:00:00
Proceeding: WS 2017 8
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

We present an optimised multi-modal dialogue agent for interactive learning
of visually grounded word meanings from a human tutor, trained on real
human-human tutoring data. Within a life-long interactive learning period, the
agent, trained using Reinforcement Learning (RL), must be able to handle
natural conversations with human users and achieve good learning performance
(accuracy) while minimising human effort in the learning process. We train and
evaluate this system in interaction with a simulated human tutor, which is
built on the BURCHAK corpus -- a Human-Human Dialogue dataset for the visual
learning task. The results show that: 1) The learned policy can coherently
interact with the simulated user to achieve the goal of the task (i.e. learning
visual attributes of objects, e.g. colour and shape); and 2) it finds a better
trade-off between classifier accuracy and tutoring costs than hand-crafted
rule-based policies, including ones with dynamic policies.
A Study of Cross-domain Generative Models applied to Cartoon Series
Authors: Eman T. Hassan, David J. Crandall
Date: 2017-09-29 00:00:00

We investigate Generative Adversarial Networks (GANs) to model one particular
kind of image: frames from TV cartoons. Cartoons are particularly interesting
because their visual appearance emphasizes the important semantic information
about a scene while abstracting out the less important details, but each
cartoon series has a distinctive artistic style that performs this abstraction
in different ways. We consider a dataset consisting of images from two popular
television cartoon series, Family Guy and The Simpsons. We examine the ability
of GANs to generate images from each of these two domains, when trained
independently as well as on both domains jointly. We find that generative
models may be capable of finding semantic-level correspondences between these
two image domains despite the unsupervised setting, even when the training data
does not give labeled alignments between them.
Strategyproof Peer Selection using Randomization, Partitioning, and Apportionment
Authors: Haris Aziz, Omer Lev, Nicholas Mattei, Jeffrey S. Rosenschein, Toby Walsh
Date: 2016-04-13 00:00:00
Tasks: Decision Making, Marketing

Peer reviews, evaluations, and selections are a fundamental aspect of modern
science. Funding bodies the world over employ experts to review and select the
best proposals from those submitted for funding. The problem of peer selection,
however, is much more general: a professional society may want to give a subset
of its members awards based on the opinions of all members; an instructor for a
Massive Open Online Course (MOOC) or an online course may want to crowdsource
grading; or a marketing company may select ideas from group brainstorming
sessions based on peer evaluation.
  We make three fundamental contributions to the study of peer selection, a
specific type of group decision-making problem, studied in computer science,
economics, and political science. First, we propose a novel mechanism that is
strategyproof, i.e., agents cannot benefit by reporting insincere valuations.
Second, we demonstrate the effectiveness of our mechanism by a comprehensive
simulation-based comparison with a suite of mechanisms found in the literature.
Finally, our mechanism employs a randomized rounding technique that is of
independent interest, as it solves the apportionment problem that arises in
various settings where discrete resources such as parliamentary representation
slots need to be divided proportionally.
Towards Universal Semantic Tagging
Authors: Lasha Abzianidze, Johan Bos
Date: 2017-09-29 00:00:00
Proceeding: WS 2017 1
Tasks: Semantic Parsing

The paper proposes the task of universal semantic tagging---tagging word
tokens with language-neutral, semantically informative tags. We argue that the
task, with its independent nature, contributes to better semantic analysis for
wide-coverage multilingual text. We present the initial version of the semantic
tagset and show that (a) the tags provide semantically fine-grained
information, and (b) they are suitable for cross-lingual semantic parsing. An
application of the semantic tagging in the Parallel Meaning Bank supports both
of these points as the tags contribute to formal lexical semantics and their
cross-lingual projection. As a part of the application, we annotate a small
corpus with the semantic tags and present new baseline result for universal
semantic tagging.
Improving speech recognition by revising gated recurrent units
Authors: Mirco Ravanelli, Philemon Brakel, Maurizio Omologo, Yoshua Bengio
Date: 2017-09-29 00:00:00
Tasks: speech-recognition, Speech Recognition

Speech recognition is largely taking advantage of deep learning, showing that
substantial benefits can be obtained by modern Recurrent Neural Networks
(RNNs). The most popular RNNs are Long Short-Term Memory (LSTMs), which
typically reach state-of-the-art performance in many tasks thanks to their
ability to learn long-term dependencies and robustness to vanishing gradients.
Nevertheless, LSTMs have a rather complex design with three multiplicative
gates, that might impair their efficient implementation. An attempt to simplify
LSTMs has recently led to Gated Recurrent Units (GRUs), which are based on just
two multiplicative gates.
  This paper builds on these efforts by further revising GRUs and proposing a
simplified architecture potentially more suitable for speech recognition. The
contribution of this work is two-fold. First, we suggest to remove the reset
gate in the GRU design, resulting in a more efficient single-gate architecture.
Second, we propose to replace tanh with ReLU activations in the state update
equations. Results show that, in our implementation, the revised architecture
reduces the per-epoch training time with more than 30% and consistently
improves recognition performance across different tasks, input features, and
noisy conditions when compared to a standard GRU.
Time Series Using Exponential Smoothing Cells
Authors: Avner Abrami, Aleksandr Y. Aravkin, Younghun Kim
Date: 2017-06-09 00:00:00
Tasks: Time Series, Time Series Analysis

Time series analysis is used to understand and predict dynamic processes,
including evolving demands in business, weather, markets, and biological
rhythms. Exponential smoothing is used in all these domains to obtain simple
interpretable models of time series and to forecast future values. Despite its
popularity, exponential smoothing fails dramatically in the presence of
outliers, large amounts of noise, or when the underlying time series changes.
  We propose a flexible model for time series analysis, using exponential
smoothing cells for overlapping time windows. The approach can detect and
remove outliers, denoise data, fill in missing observations, and provide
meaningful forecasts in challenging situations. In contrast to classic
exponential smoothing, which solves a nonconvex optimization problem over the
smoothing parameters and initial state, the proposed approach requires solving
a single structured convex optimization problem. Recent developments in
efficient convex optimization of large-scale dynamic models make the approach
tractable. We illustrate new capabilities using synthetic examples, and then
use the approach to analyze and forecast noisy real-world time series. Code for
the approach and experiments is publicly available.
De-identification of medical records using conditional random fields and long short-term memory networks
Authors: Zhipeng Jiang, Chao Zhao, Bin He, Yi Guan, Jingchi Jiang
Date: 2017-09-20 00:00:00
Tasks: De-identification, Sentence

The CEGS N-GRID 2016 Shared Task 1 in Clinical Natural Language Processing
focuses on the de-identification of psychiatric evaluation records. This paper
describes two participating systems of our team, based on conditional random
fields (CRFs) and long short-term memory networks (LSTMs). A pre-processing
module was introduced for sentence detection and tokenization before
de-identification. For CRFs, manually extracted rich features were utilized to
train the model. For LSTMs, a character-level bi-directional LSTM network was
applied to represent tokens and classify tags for each token, following which a
decoding layer was stacked to decode the most probable protected health
information (PHI) terms. The LSTM-based system attained an i2b2 strict
micro-F_1 measure of 89.86%, which was higher than that of the CRF-based
system.
IQ of Neural Networks
Authors: Dokhyam Hoshen, Michael Werman
Date: 2017-09-29 00:00:00

IQ tests are an accepted method for assessing human intelligence. The tests
consist of several parts that must be solved under a time constraint. Of all
the tested abilities, pattern recognition has been found to have the highest
correlation with general intelligence. This is primarily because pattern
recognition is the ability to find order in a noisy environment, a necessary
skill for intelligent agents. In this paper, we propose a convolutional neural
network (CNN) model for solving geometric pattern recognition problems. The CNN
receives as input multiple ordered input images and outputs the next image
according to the pattern. Our CNN is able to solve problems involving rotation,
reflection, color, size and shape patterns and score within the top 5% of human
performance.
A Graphical Social Topology Model for Multi-Object Tracking
Authors: Shan Gao, Xiaogang Chen, Qixiang Ye, Junliang Xing, Arjan Kuijper, Xiangyang Ji
Date: 2017-02-14 00:00:00
Tasks: Multi-Object Tracking, Object, Object Tracking

Tracking multiple objects is a challenging task when objects move in groups
and occlude each other. Existing methods have investigated the problems of
group division and group energy-minimization; however, lacking overall
object-group topology modeling limits their ability in handling complex object
and group dynamics. Inspired with the social affinity property of moving
objects, we propose a Graphical Social Topology (GST) model, which estimates
the group dynamics by jointly modeling the group structure and the states of
objects using a topological representation. With such topology representation,
moving objects are not only assigned to groups, but also dynamically connected
with each other, which enables in-group individuals to be correctly associated
and the cohesion of each group to be precisely modeled. Using well-designed
topology learning modules and topology training, we infer the birth/death and
merging/splitting of dynamic groups. With the GST model, the proposed
multi-object tracker can naturally facilitate the occlusion problem by treating
the occluded object and other in-group members as a whole while leveraging
overall state transition. Experiments on both RGB and RGB-D datasets confirm
that the proposed multi-object tracker improves the state-of-the-arts
especially in crowded scenes.
Interacting Conceptual Spaces I : Grammatical Composition of Concepts
Authors: Joe Bolt, Bob Coecke, Fabrizio Genovese, Martha Lewis, Dan Marsden, Robin Piedeleu
Date: 2017-03-24 00:00:00

The categorical compositional approach to meaning has been successfully
applied in natural language processing, outperforming other models in
mainstream empirical language processing tasks. We show how this approach can
be generalized to conceptual space models of cognition. In order to do this,
first we introduce the category of convex relations as a new setting for
categorical compositional semantics, emphasizing the convex structure important
to conceptual space applications. We then show how to construct conceptual
spaces for various types such as nouns, adjectives and verbs. Finally we show
by means of examples how concepts can be systematically combined to establish
the meanings of composite phrases from the meanings of their constituent parts.
This provides the mathematical underpinnings of a new compositional approach to
cognition.
A Nonlinear Orthogonal Non-Negative Matrix Factorization Approach to Subspace Clustering
Authors: Dijana Tolic, Nino Antulov-Fantulin, Ivica Kopriva
Date: 2017-09-29 00:00:00
Tasks: Clustering

A recent theoretical analysis shows the equivalence between non-negative
matrix factorization (NMF) and spectral clustering based approach to subspace
clustering. As NMF and many of its variants are essentially linear, we
introduce a nonlinear NMF with explicit orthogonality and derive general
kernel-based orthogonal multiplicative update rules to solve the subspace
clustering problem. In nonlinear orthogonal NMF framework, we propose two
subspace clustering algorithms, named kernel-based non-negative subspace
clustering KNSC-Ncut and KNSC-Rcut and establish their connection with spectral
normalized cut and ratio cut clustering. We further extend the nonlinear
orthogonal NMF framework and introduce a graph regularization to obtain a
factorization that respects a local geometric structure of the data after the
nonlinear mapping. The proposed NMF-based approach to subspace clustering takes
into account the nonlinear nature of the manifold, as well as its intrinsic
local geometry, which considerably improves the clustering performance when
compared to the several recently proposed state-of-the-art methods.
Privacy Preserving Identification Using Sparse Approximation with Ambiguization
Authors: Behrooz Razeghi, Slava Voloshynovskiy, Dimche Kostadinov, Olga Taran
Date: 2017-09-29 00:00:00
Tasks: Privacy Preserving

In this paper, we consider a privacy preserving encoding framework for
identification applications covering biometrics, physical object security and
the Internet of Things (IoT). The proposed framework is based on a sparsifying
transform, which consists of a trained linear map, an element-wise
nonlinearity, and privacy amplification. The sparsifying transform and privacy
amplification are not symmetric for the data owner and data user. We
demonstrate that the proposed approach is closely related to sparse ternary
codes (STC), a recent information-theoretic concept proposed for fast
approximate nearest neighbor (ANN) search in high dimensional feature spaces
that being machine learning in nature also offers significant benefits in
comparison to sparse approximation and binary embedding approaches. We
demonstrate that the privacy of the database outsourced to a server as well as
the privacy of the data user are preserved at a low computational cost, storage
and communication burdens.
Deep Competitive Pathway Networks
Authors: Jia-Ren Chang, Yong-Sheng Chen
Date: 2017-09-29 00:00:00
Tasks: Image Classification, Object Recognition

In the design of deep neural architectures, recent studies have demonstrated
the benefits of grouping subnetworks into a larger network. For examples, the
Inception architecture integrates multi-scale subnetworks and the residual
network can be regarded that a residual unit combines a residual subnetwork
with an identity shortcut. In this work, we embrace this observation and
propose the Competitive Pathway Network (CoPaNet). The CoPaNet comprises a
stack of competitive pathway units and each unit contains multiple parallel
residual-type subnetworks followed by a max operation for feature competition.
This mechanism enhances the model capability by learning a variety of features
in subnetworks. The proposed strategy explicitly shows that the features
propagate through pathways in various routing patterns, which is referred to as
pathway encoding of category information. Moreover, the cross-block shortcut
can be added to the CoPaNet to encourage feature reuse. We evaluated the
proposed CoPaNet on four object recognition benchmarks: CIFAR-10, CIFAR-100,
SVHN, and ImageNet. CoPaNet obtained the state-of-the-art or comparable results
using similar amounts of parameters. The code of CoPaNet is available at:
https://github.com/JiaRenChang/CoPaNet.
Sensor Synthesis for POMDPs with Reachability Objectives
Authors: Krishnendu Chatterjee, Martin Chmelik, Ufuk Topcu
Date: 2017-09-29 00:00:00

Partially observable Markov decision processes (POMDPs) are widely used in
probabilistic planning problems in which an agent interacts with an environment
using noisy and imprecise sensors. We study a setting in which the sensors are
only partially defined and the goal is to synthesize "weakest" additional
sensors, such that in the resulting POMDP, there is a small-memory policy for
the agent that almost-surely (with probability~1) satisfies a reachability
objective. We show that the problem is NP-complete, and present a symbolic
algorithm by encoding the problem into SAT instances. We illustrate trade-offs
between the amount of memory of the policy and the number of additional sensors
on a simple example. We have implemented our approach and consider three
classical POMDP examples from the literature, and show that in all the examples
the number of sensors can be significantly decreased (as compared to the
existing solutions in the literature) without increasing the complexity of the
policies.