pwc-archive/papers-with-abstracts huggingface dataset:
Lookup Table-Based Consensus Algorithm for Real-Time Longitudinal Motion Control of Connected and Automated Vehicles
Authors: Ziran Wang, Kyuntae Han, BaekGyu Kim, Guoyuan Wu, Matthew J. Barth
Date: 2019-02-20 00:00:00

Connected and automated vehicle (CAV) technology is one of the promising solutions to addressing the safety, mobility and sustainability issues of our current transportation systems. Specifically, the control algorithm plays an important role in a CAV system, since it executes the commands generated by former steps, such as communication, perception, and planning. In this study, we propose a consensus algorithm to control the longitudinal motion of CAVs in real time. Different from previous studies in this field where control gains of the consensus algorithm are pre-determined and fixed, we develop algorithms to build up a lookup table, searching for the ideal control gains with respect to different initial conditions of CAVs in real time. Numerical simulation shows that, the proposed lookup table-based consensus algorithm outperforms the authors' previous work, as well as van Arem's linear feedback-based longitudinal motion control algorithm in all four different scenarios with various initial conditions of CAVs, in terms of convergence time and maximum jerk of the simulation run.
Predicting ConceptNet Path Quality Using Crowdsourced Assessments of Naturalness
Authors: Yilun Zhou, Steven Schockaert, Julie A. Shah
Date: 2019-02-21 00:00:00
Tasks: Knowledge Graphs

In many applications, it is important to characterize the way in which two
concepts are semantically related. Knowledge graphs such as ConceptNet provide
a rich source of information for such characterizations by encoding relations
between concepts as edges in a graph. When two concepts are not directly
connected by an edge, their relationship can still be described in terms of the
paths that connect them. Unfortunately, many of these paths are uninformative
and noisy, which means that the success of applications that use such path
features crucially relies on their ability to select high-quality paths. In
existing applications, this path selection process is based on relatively
simple heuristics. In this paper we instead propose to learn to predict path
quality from crowdsourced human assessments. Since we are interested in a
generic task-independent notion of quality, we simply ask human participants to
rank paths according to their subjective assessment of the paths' naturalness,
without attempting to define naturalness or steering the participants towards
particular indicators of quality. We show that a neural network model trained
on these assessments is able to predict human judgments on unseen paths with
near optimal performance. Most notably, we find that the resulting path
selection method is substantially better than the current heuristic approaches
at identifying meaningful paths.
ntuer at SemEval-2019 Task 3: Emotion Classification with Word and Sentence Representations in RCNN
Authors: Peixiang Zhong, Chunyan Miao
Date: 2019-02-21 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Emotion Classification, General Classification, Sensitivity, Sentence

In this paper we present our model on the task of emotion detection in
textual conversations in SemEval-2019. Our model extends the Recurrent
Convolutional Neural Network (RCNN) by using external fine-tuned word
representations and DeepMoji sentence representations. We also explored several
other competitive pre-trained word and sentence representations including ELMo,
BERT and InferSent but found inferior performance. In addition, we conducted
extensive sensitivity analysis, which empirically shows that our model is
relatively robust to hyper-parameters. Our model requires no handcrafted
features or emotion lexicons but achieved good performance with a micro-F1
score of 0.7463.
Learning Dual Retrieval Module for Semi-supervised Relation Extraction
Authors: Hongtao Lin, Jun Yan, Meng Qu, Xiang Ren
Date: 2019-02-20 00:00:00
Tasks: MULTI-VIEW LEARNING, Relation, Relation Extraction, Relation Prediction, Retrieval, Sentence

Relation extraction is an important task in structuring content of text data,
and becomes especially challenging when learning with weak supervision---where
only a limited number of labeled sentences are given and a large number of
unlabeled sentences are available. Most existing work exploits unlabeled data
based on the ideas of self-training (i.e., bootstrapping a model) and
multi-view learning (e.g., ensembling multiple model variants). However, these
methods either suffer from the issue of semantic drift, or do not fully capture
the problem characteristics of relation extraction. In this paper, we leverage
a key insight that retrieving sentences expressing a relation is a dual task of
predicting relation label for a given sentence---two tasks are complementary to
each other and can be optimized jointly for mutual enhancement. To model this
intuition, we propose DualRE, a principled framework that introduces a
retrieval module which is jointly trained with the original relation prediction
module. In this way, high-quality samples selected by retrieval module from
unlabeled data can be used to improve prediction module, and vice versa.
Experimental results\footnote{\small Code and data can be found at
\url{https://github.com/INK-USC/DualRE}.} on two public datasets as well as
case studies demonstrate the effectiveness of the DualRE approach.
Mixture Models for Diverse Machine Translation: Tricks of the Trade
Authors: Tianxiao Shen, Myle Ott, Michael Auli, Marc'Aurelio Ranzato
Date: 2019-02-20 00:00:00
Tasks: Machine Translation, Text Generation, Translation

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
Deep Speaker Embedding Learning with Multi-Level Pooling for Text-Independent Speaker Verification
Authors: Yun Tang, Guohong Ding, Jing Huang, Xiaodong He, Bo-Wen Zhou
Date: 2019-02-21 00:00:00
Tasks: Speaker Verification, Text-Independent Speaker Verification

This paper aims to improve the widely used deep speaker embedding x-vector
model. We propose the following improvements: (1) a hybrid neural network
structure using both time delay neural network (TDNN) and long short-term
memory neural networks (LSTM) to generate complementary speaker information at
different levels; (2) a multi-level pooling strategy to collect speaker
information from both TDNN and LSTM layers; (3) a regularization scheme on the
speaker embedding extraction layer to make the extracted embeddings suitable
for the following fusion step. The synergy of these improvements are shown on
the NIST SRE 2016 eval test (with a 19% EER reduction) and SRE 2018 dev test
(with a 9% EER reduction), as well as more than 10% DCF scores reduction on
these two test sets over the x-vector baseline.
Pretrained language model transfer on neural named entity recognition in Indonesian conversational texts
Authors: Rezka Leonandya, Fariz Ikhwantri
Date: 2019-02-21 00:00:00
Tasks: Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Small Data Image Classification, Transfer Learning

Named entity recognition (NER) is an important task in NLP, which is all the
more challenging in conversational domain with their noisy facets. Moreover,
conversational texts are often available in limited amount, making supervised
tasks infeasible. To learn from small data, strong inductive biases are
required. Previous work relied on hand-crafted features to encode these biases
until transfer learning emerges. Here, we explore a transfer learning method,
namely language model pretraining, on NER task in Indonesian conversational
texts. We utilize large unlabeled data (generic domain) to be transferred to
conversational texts, enabling supervised training on limited in-domain data.
We report two transfer learning variants, namely supervised model fine-tuning
and unsupervised pretrained LM fine-tuning. Our experiments show that both
variants outperform baseline neural models when trained on small data (100
sentences), yielding an absolute improvement of 32 points of test F1 score.
Furthermore, we find that the pretrained LM encodes part-of-speech information
which is a strong predictor for NER.
Deep Short Text Classification with Knowledge Powered Attention
Authors: Jindong Chen, Yizhou Hu, Jingping Liu, Yanghua Xiao, Haiyun Jiang
Date: 2019-02-21 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification

Short text classification is one of important tasks in Natural Language
Processing (NLP). Unlike paragraphs or documents, short texts are more
ambiguous since they have not enough contextual information, which poses a
great challenge for classification. In this paper, we retrieve knowledge from
external knowledge source to enhance the semantic representation of short
texts. We take conceptual information as a kind of knowledge and incorporate it
into deep neural networks. For the purpose of measuring the importance of
knowledge, we introduce attention mechanisms and propose deep Short Text
Classification with Knowledge powered Attention (STCKA). We utilize Concept
towards Short Text (C- ST) attention and Concept towards Concept Set (C-CS)
attention to acquire the weight of concepts from two aspects. And we classify a
short text with the help of conceptual information. Unlike traditional
approaches, our model acts like a human being who has intrinsic ability to make
decisions based on observation (i.e., training data for machines) and pays more
attention to important knowledge. We also conduct extensive experiments on four
public datasets for different tasks. The experimental results and case studies
show that our model outperforms the state-of-the-art methods, justifying the
effectiveness of knowledge powered attention.
Development of a classifiers/quantifiers dictionary towards French-Japanese MT
Authors: Mutsuko Tomokiyo, Mathieu Mangeot, Christian Boitet
Date: 2019-02-21 00:00:00
Proceeding: MTSummit 2017 9
Tasks: Machine Translation, Translation

Although classifiers/quantifiers (CQs) expressions appear frequently in
everyday communications or written documents, they are described neither in
classical bilingual paper dictionaries , nor in machine-readable dictionaries.
The paper describes a CQs dictionary, edited from the corpus we have annotated,
and its usage in the framework of French-Japanese machine translation (MT). CQs
treatment in MT often causes problems of lexical ambiguity, polylexical phrase
recognition difficulties in analysis and doubtful output in
transfer-generation, in particular for distant languages pairs like French and
Japanese. Our basic treatment of CQs is to annotate the corpus by UNL-UWs
(Universal Networking Language-Universal words) 1 , and then to produce a
bilingual or multilingual dictionary of CQs, based on synonymy through identity
of UWs.
Audio-Linguistic Embeddings for Spoken Sentences
Authors: Albert Haque, Michelle Guo, Prateek Verma, Li Fei-Fei
Date: 2019-02-20 00:00:00
Tasks: Decoder, Emotion Recognition, Sentence, Sentence Embeddings, speech-recognition, Speech Recognition, Spoken Language Understanding

We propose spoken sentence embeddings which capture both acoustic and
linguistic content. While existing works operate at the character, phoneme, or
word level, our method learns long-term dependencies by modeling speech at the
sentence level. Formulated as an audio-linguistic multitask learning problem,
our encoder-decoder model simultaneously reconstructs acoustic and natural
language features from audio. Our results show that spoken sentence embeddings
outperform phoneme and word-level baselines on speech recognition and emotion
recognition tasks. Ablation studies show that our embeddings can better model
high-level acoustic concepts while retaining linguistic content. Overall, our
work illustrates the viability of generic, multi-modal sentence embeddings for
spoken language understanding.
Active online learning in the binary perceptron problem
Authors: Hai-Jun Zhou
Date: 2019-02-21 00:00:00

The binary perceptron is the simplest artificial neural network formed by $N$
input units and one output unit, with the neural states and the synaptic
weights all restricted to $\pm 1$ values. The task in the teacher--student
scenario is to infer the hidden weight vector by training on a set of labeled
patterns. Previous efforts on the passive learning mode have shown that
learning from independent random patterns is quite inefficient. Here we
consider the active online learning mode in which the student designs every new
Ising training pattern. We demonstrate that it is mathematically possible to
achieve perfect (error-free) inference using only $N$ designed training
patterns, but this is computationally unfeasible for large systems. We then
investigate two Bayesian statistical designing protocols, which require $2.3 N$
and $1.9 N$ training patterns, respectively, to achieve error-free inference.
If the training patterns are instead designed through deductive reasoning,
perfect inference is achieved using $N\!+\!\log_{2}\!N$ samples. The
performance gap between Bayesian and deductive designing strategies may be
shortened in future work by taking into account the possibility of ergodicity
breaking in the version space of the binary perceptron.
Gradient Scheduling with Global Momentum for Non-IID Data Distributed Asynchronous Training
Authors: Chengjie Li, Ruixuan Li, Haozhao Wang, Yuhua Li, Pan Zhou, Song Guo, Keqin Li
Date: 2019-02-21 00:00:00
Tasks: Scheduling

Distributed asynchronous offline training has received widespread attention in recent years because of its high performance on large-scale data and complex models. As data are distributed from cloud-centric to edge nodes, a big challenge for distributed machine learning systems is how to handle native and natural non-independent and identically distributed (non-IID) data for training. Previous asynchronous training methods do not have a satisfying performance on non-IID data because it would result in that the training process fluctuates greatly which leads to an abnormal convergence. We propose a gradient scheduling algorithm with partly averaged gradients and global momentum (GSGM) for non-IID data distributed asynchronous training. Our key idea is to apply global momentum and local average to the biased gradient after scheduling, in order to make the training process steady. Experimental results show that for non-IID data training under the same experimental conditions, GSGM on popular optimization algorithms can achieve a 20% increase in training stability with a slight improvement in accuracy on Fashion-Mnist and CIFAR-10 datasets. Meanwhile, when expanding distributed scale on CIFAR-100 dataset that results in sparse data distribution, GSGM can perform a 37% improvement on training stability. Moreover, only GSGM can converge well when the number of computing nodes grows to 30, compared to the state-of-the-art distributed asynchronous algorithms. At the same time, GSGM is robust to different degrees of non-IID data.
All-neural online source separation, counting, and diarization for meeting analysis
Authors: Thilo von Neumann, Keisuke Kinoshita, Marc Delcroix, Shoko Araki, Tomohiro Nakatani, Reinhold Haeb-Umbach
Date: 2019-02-21 00:00:00
Tasks: All, Automatic Speech Recognition, Automatic Speech Recognition (ASR), speaker-diarization, Speaker Diarization, speech-recognition, Speech Recognition

Automatic meeting analysis comprises the tasks of speaker counting, speaker
diarization, and the separation of overlapped speech, followed by automatic
speech recognition. This all has to be carried out on arbitrarily long sessions
and, ideally, in an online or block-online manner. While significant progress
has been made on individual tasks, this paper presents for the first time an
all-neural approach to simultaneous speaker counting, diarization and source
separation. The NN-based estimator operates in a block-online fashion and
tracks speakers even if they remain silent for a number of time blocks, thus
learning a stable output order for the separated sources. The neural network is
recurrent over time as well as over the number of sources. The simulation
experiments show that state of the art separation performance is achieved,
while at the same time delivering good diarization and source counting results.
It even generalizes well to an unseen large number of blocks.
Deep Adaptive Input Normalization for Time Series Forecasting
Authors: Nikolaos Passalis, Anastasios Tefas, Juho Kanniainen, Moncef Gabbouj, Alexandros Iosifidis
Date: 2019-02-21 00:00:00
Tasks: Load Forecasting, Time Series, Time Series Analysis, Time Series Forecasting

Deep Learning (DL) models can be used to tackle time series analysis tasks with great success. However, the performance of DL models can degenerate rapidly if the data are not appropriately normalized. This issue is even more apparent when DL is used for financial time series forecasting tasks, where the non-stationary and multimodal nature of the data pose significant challenges and severely affect the performance of DL models. In this work, a simple, yet effective, neural layer, that is capable of adaptively normalizing the input time series, while taking into account the distribution of the data, is proposed. The proposed layer is trained in an end-to-end fashion using back-propagation and leads to significant performance improvements compared to other evaluated normalization schemes. The proposed method differs from traditional normalization methods since it learns how to perform normalization for a given task instead of using a fixed normalization scheme. At the same time, it can be directly applied to any new time series without requiring re-training. The effectiveness of the proposed method is demonstrated using a large-scale limit order book dataset, as well as a load forecasting dataset.
Error bounds for approximations with deep ReLU neural networks in $W^{s,p}$ norms
Authors: Ingo Gühring, Gitta Kutyniok, Philipp Petersen
Date: 2019-02-21 00:00:00

We analyze approximation rates of deep ReLU neural networks for
Sobolev-regular functions with respect to weaker Sobolev norms. First, we
construct, based on a calculus of ReLU networks, artificial neural networks
with ReLU activation functions that achieve certain approximation rates.
Second, we establish lower bounds for the approximation by ReLU neural networks
for classes of Sobolev-regular functions. Our results extend recent advances in
the approximation theory of ReLU networks to the regime that is most relevant
for applications in the numerical analysis of partial differential equations.
Continuous Outlier Mining of Streaming Data in Flink
Authors: Theodoros Toliopoulos, Anastasios Gounaris, Kostas Tsichlas, Apostolos Papadopoulos, Sandra Sampaio
Date: 2019-02-21 00:00:00

In this work, we focus on distance-based outliers in a metric space, where
the status of an entity as to whether it is an outlier is based on the number
of other entities in its neighborhood. In recent years, several solutions have
tackled the problem of distance-based outliers in data streams, where outliers
must be mined continuously as new elements become available. An interesting
research problem is to combine the streaming environment with massively
parallel systems to provide scalable streambased algorithms. However, none of
the previously proposed techniques refer to a massively parallel setting. Our
proposal fills this gap and investigates the challenges in transferring
state-of-the-art techniques to Apache Flink, a modern platform for intensive
streaming analytics. We thoroughly present the technical challenges encountered
and the alternatives that may be applied. We show speed-ups of up to 117 (resp.
2076) times over a naive parallel (resp. non-parallel) solution in Flink, by
using just an ordinary four-core machine and a real-world dataset. When moving
to a three-machine cluster, due to less contention, we manage to achieve both
better scalability in terms of the window slide size and the data
dimensionality, and even higher speed-ups, e.g., by a factor of 510. Overall,
our results demonstrate that oulier mining can be achieved in an efficient and
scalable manner. The resulting techniques have been made publicly available as
open-source software.
Short-distance commuters in the smart city
Authors: Francisco Benita, Garvit Bansal, Georgios Piliouras, Bige Tunçer
Date: 2019-02-16 00:00:00

This study models and examines commuter's preferences for short-distance
transportation modes, namely: walking, taking a bus or riding a metro. It is
used a unique dataset from a large-scale field experiment in Singapore that
provides rich information about tens of thousands of commuters' behavior. In
contrast to the standard approach, this work does not relay on survey data.
Conversely, the chosen transportation modes are identified by processing raw
data (latitude, longitude, timestamp). The approach of this work exploits the
information generated by the smart transportation system in the city that make
suitable the task of obtaining granular and nearly real-time data. Novel
algorithms are proposed with the intention to generate proxies for walkability
and public transport attributes. The empirical results of the case study
suggest that commuters do no differentiate between public transport choices
(bus and metro), therefore possible nested structures for the public transport
modes are rejected.
A Mean Field Theory of Batch Normalization
Authors: Greg Yang, Jeffrey Pennington, Vinay Rao, Jascha Sohl-Dickstein, Samuel S. Schoenholz
Date: 2019-02-21 00:00:00
Proceeding: ICLR 2019 5

We develop a mean field theory for batch normalization in fully-connected
feedforward neural networks. In so doing, we provide a precise characterization
of signal propagation and gradient backpropagation in wide batch-normalized
networks at initialization. Our theory shows that gradient signals grow
exponentially in depth and that these exploding gradients cannot be eliminated
by tuning the initial weight variances or by adjusting the nonlinear activation
function. Indeed, batch normalization itself is the cause of gradient
explosion. As a result, vanilla batch-normalized networks without skip
connections are not trainable at large depths for common initialization
schemes, a prediction that we verify with a variety of empirical simulations.
While gradient explosion cannot be eliminated, it can be reduced by tuning the
network close to the linear regime, which improves the trainability of deep
batch-normalized networks without residual connections. Finally, we investigate
the learning dynamics of batch-normalized networks and observe that after a
single step of optimization the networks achieve a relatively stable
equilibrium in which gradients have dramatically smaller dynamic range. Our
theory leverages Laplace, Fourier, and Gegenbauer transforms and we derive new
identities that may be of independent interest.
Mitigating Metaphors: A Comprehensible Guide to Recent Nature-Inspired Algorithms
Authors: Michael Adam Lones
Date: 2019-02-21 00:00:00
Tasks: Evolutionary Algorithms

In recent years, a plethora of new metaheuristic algorithms have explored different sources of inspiration within the biological and natural worlds. This nature-inspired approach to algorithm design has been widely criticised. A notable issue is the tendency for authors to use terminology that is derived from the domain of inspiration, rather than the broader domains of metaheuristics and optimisation. This makes it difficult to both comprehend how these algorithms work and understand their relationships to other metaheuristics. This paper attempts to address this issue, at least to some extent, by providing accessible descriptions of the most cited nature-inspired algorithms published in the last twenty years. It also discusses commonalities between these algorithms and more classical nature-inspired metaheuristics such as evolutionary algorithms and particle swarm optimisation, and finishes with a discussion of future directions for the field.