pwc-archive/papers-with-abstracts huggingface dataset:
A Tempt to Unify Heterogeneous Driving Databases using Traffic Primitives
Authors: Jiacheng Zhu, Wenshuo Wang, Ding Zhao
Date: 2018-05-13 00:00:00
Tasks: Autonomous Vehicles, Time Series, Time Series Analysis

A multitude of publicly-available driving datasets and data platforms have
been raised for autonomous vehicles (AV). However, the heterogeneities of
databases in size, structure and driving context make existing datasets
practically ineffective due to a lack of uniform frameworks and searchable
indexes. In order to overcome these limitations on existing public datasets,
this paper proposes a data unification framework based on traffic primitives
with ability to automatically unify and label heterogeneous traffic data. This
is achieved by two steps: 1) Carefully arrange raw multidimensional time series
driving data into a relational database and then 2) automatically extract
labeled and indexed traffic primitives from traffic data through a Bayesian
nonparametric learning method. Finally, we evaluate the effectiveness of our
developed framework using the collected real vehicle data.
Emergence and Evolution of Hierarchical Structure in Complex Systems
Authors: Payam Siyari, Bistra Dilkina, Constantine Dovrolis
Date: 2018-05-13 00:00:00

It is well known that many complex systems, both in technology and nature,
exhibit hierarchical modularity: smaller modules, each of them providing a
certain function, are used within larger modules that perform more complex
functions. What is not well understood however is how this hierarchical
structure (which is fundamentally a network property) emerges, and how it
evolves over time. We propose a modeling framework, referred to as Evo-Lexis,
that provides insight to some fundamental questions about evolving hierarchical
systems. Evo-Lexis models the most elementary modules of the system as symbols
("sources") and the modules at the highest level of the hierarchy as sequences
of those symbols ("targets"). Evo-Lexis computes the optimized adjustment of a
given hierarchy when the set of targets changes over time by additions and
removals (a process referred to as "incremental design"). In this paper we use
computation modeling to show that:
  - Low-cost and deep hierarchies emerge when the population of target
sequences evolves through tinkering and mutation. - Strong selection on the
cost of new candidate targets results in reuse of more complex (longer) nodes
in an optimized hierarchy. - The bias towards reuse of complex nodes results in
an "hourglass architecture" (i.e., few intermediate nodes that cover almost all
source-target paths). - With such bias, the core nodes are conserved for
relatively long time periods although still being vulnerable to major
transitions and punctuated equilibria. - Finally, we analyze the differences in
terms of cost and structure between incrementally designed hierarchies and the
corresponding "clean-slate" hierarchies which result when the system is
designed from scratch after a change.
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
Authors: Arun Mallya, Svetlana Lazebnik
Date: 2017-11-15 00:00:00
Proceeding: CVPR 2018 6
Tasks: Continual Learning, Network Pruning

This paper presents a method for adding multiple tasks to a single deep
neural network while avoiding catastrophic forgetting. Inspired by network
pruning techniques, we exploit redundancies in large deep networks to free up
parameters that can then be employed to learn new tasks. By performing
iterative pruning and network re-training, we are able to sequentially "pack"
multiple tasks into a single network while ensuring minimal drop in performance
and minimal storage overhead. Unlike prior work that uses proxy losses to
maintain accuracy on older tasks, we always optimize for the task at hand. We
perform extensive experiments on a variety of network architectures and
large-scale datasets, and observe much better robustness against catastrophic
forgetting than prior work. In particular, we are able to add three
fine-grained classification tasks to a single ImageNet-trained VGG-16 network
and achieve accuracies close to those of separately trained networks for each
task. Code available at https://github.com/arunmallya/packnet
Extendable Neural Matrix Completion
Authors: Duc Minh Nguyen, Evaggelia Tsiligianni, Nikos Deligiannis
Date: 2018-05-13 00:00:00
Tasks: Matrix Completion

Matrix completion is one of the key problems in signal processing and machine
learning, with applications ranging from image pro- cessing and data gathering
to classification and recommender sys- tems. Recently, deep neural networks
have been proposed as la- tent factor models for matrix completion and have
achieved state- of-the-art performance. Nevertheless, a major problem with
existing neural-network-based models is their limited capabilities to extend to
samples unavailable at the training stage. In this paper, we propose a deep
two-branch neural network model for matrix completion. The proposed model not
only inherits the predictive power of neural net- works, but is also capable of
extending to partially observed samples outside the training set, without the
need of retraining or fine-tuning. Experimental studies on popular movie rating
datasets prove the ef- fectiveness of our model compared to the state of the
art, in terms of both accuracy and extendability.
On the Practical Computational Power of Finite Precision RNNs for Language Recognition
Authors: Gail Weiss, Yoav Goldberg, Eran Yahav
Date: 2018-05-13 00:00:00
Proceeding: ACL 2018 7

While Recurrent Neural Networks (RNNs) are famously known to be Turing
complete, this relies on infinite precision in the states and unbounded
computation time. We consider the case of RNNs with finite precision whose
computation time is linear in the input length. Under these limitations, we
show that different RNN variants have different computational power. In
particular, we show that the LSTM and the Elman-RNN with ReLU activation are
strictly stronger than the RNN with a squashing activation and the GRU. This is
achieved because LSTMs and ReLU-RNNs can easily implement counting behavior. We
show empirically that the LSTM does indeed learn to effectively use the
counting mechanism.
Comprehensive Supersense Disambiguation of English Prepositions and Possessives
Authors: Nathan Schneider, Jena D. Hwang, Vivek Srikumar, Jakob Prange, Austin Blodgett, Sarah R. Moeller, Aviram Stern, Adi Bitan, Omri Abend
Date: 2018-05-13 00:00:00
Proceeding: ACL 2018 7
Tasks: Natural Language Understanding

Semantic relations are often signaled with prepositional or possessive
marking--but extreme polysemy bedevils their analysis and automatic
interpretation. We introduce a new annotation scheme, corpus, and task for the
disambiguation of prepositions and possessives in English. Unlike previous
approaches, our annotations are comprehensive with respect to types and tokens
of these markers; use broadly applicable supersense classes rather than
fine-grained dictionary definitions; unite prepositions and possessives under
the same class inventory; and distinguish between a marker's lexical
contribution and the role it marks in the context of a predicate or scene.
Strong interannotator agreement rates, as well as encouraging disambiguation
results with established supervised methods, speak to the viability of the
scheme and task.
Personalizing Dialogue Agents: I have a dog, do you have pets too?
Authors: Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, Jason Weston
Date: 2018-01-22 00:00:00
Proceeding: ACL 2018 7
Tasks: Conversational Response Selection, Dialogue Generation, Specificity

Chit-chat models are known to have several problems: they lack specificity,
do not display a consistent personality and are often not very captivating. In
this work we present the task of making chit-chat more engaging by conditioning
on profile information. We collect data and train models to (i) condition on
their given profile information; and (ii) information about the person they are
talking to, resulting in improved dialogues, as measured by next utterance
prediction. Since (ii) is initially unknown our model is trained to engage its
partner with personal topics, and we show the resulting dialogue can be used to
predict profile information about the interlocutors.
General solutions for nonlinear differential equations: a rule-based self-learning approach using deep reinforcement learning
Authors: Shiyin Wei, Xiaowei Jin, Hui Li
Date: 2018-05-13 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning, Self-Learning, Transfer Learning

A universal rule-based self-learning approach using deep reinforcement learning (DRL) is proposed for the first time to solve nonlinear ordinary differential equations and partial differential equations. The solver consists of a deep neural network-structured actor that outputs candidate solutions, and a critic derived only from physical rules (governing equations and boundary and initial conditions). Solutions in discretized time are treated as multiple tasks sharing the same governing equation, and the current step parameters provide an ideal initialization for the next owing to the temporal continuity of the solutions, which shows a transfer learning characteristic and indicates that the DRL solver has captured the intrinsic nature of the equation. The approach is verified through solving the Schr\"odinger, Navier-Stokes, Burgers', Van der Pol, and Lorenz equations and an equation of motion. The results indicate that the approach gives solutions with high accuracy, and the solution process promises to get faster.
Nonparametric regression using deep neural networks with ReLU activation function
Authors: Johannes Schmidt-Hieber
Date: 2017-08-22 00:00:00
Tasks: Additive models, regression

Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to $\log n$-factors) under a general composition assumption on the regression function. The framework includes many well-studied structural constraints such as (generalized) additive models. While there is a lot of flexibility in the network architecture, the tuning parameter is the sparsity of the network. Specifically, we consider large networks with number of potential network parameters exceeding the sample size. The analysis gives some insights into why multilayer feedforward neural networks perform well in practice. Interestingly, for ReLU activation function the depth (number of layers) of the neural network architectures plays an important role and our theory suggests that for nonparametric regression, scaling the network depth with the sample size is natural. It is also shown that under the composition assumption wavelet estimators can only achieve suboptimal rates.
Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering
Authors: Rui Zhang, Cicero Nogueira dos santos, Michihiro Yasunaga, Bing Xiang, Dragomir Radev
Date: 2018-05-13 00:00:00
Proceeding: ACL 2018 7
Tasks: Clustering, coreference-resolution, Coreference Resolution

Coreference resolution aims to identify in a text all mentions that refer to
the same real-world entity. The state-of-the-art end-to-end neural coreference
model considers all text spans in a document as potential mentions and learns
to link an antecedent for each possible mention. In this paper, we propose to
improve the end-to-end coreference resolution system by (1) using a biaffine
attention model to get antecedent scores for each possible mention, and (2)
jointly optimizing the mention detection accuracy and the mention clustering
log-likelihood given the mention cluster labels. Our model achieves the
state-of-the-art performance on the CoNLL-2012 Shared Task English test set.
An Optimal Rewiring Strategy for Reinforcement Social Learning in Cooperative Multiagent Systems
Authors: Hongyao Tang, Li Wang, Zan Wang, Tim Baarslag, Jianye Hao
Date: 2018-05-13 00:00:00

Multiagent coordination in cooperative multiagent systems (MASs) has been
widely studied in both fixed-agent repeated interaction setting and the static
social learning framework. However, two aspects of dynamics in real-world
multiagent scenarios are currently missing in existing works. First, the
network topologies can be dynamic where agents may change their connections
through rewiring during the course of interactions. Second, the game matrix
between each pair of agents may not be static and usually not known as a prior.
Both the network dynamic and game uncertainty increase the coordination
difficulty among agents. In this paper, we consider a multiagent dynamic social
learning environment in which each agent can choose to rewire potential
partners and interact with randomly chosen neighbors in each round. We propose
an optimal rewiring strategy for agents to select most beneficial peers to
interact with for the purpose of maximizing the accumulated payoff in repeated
interactions. We empirically demonstrate the effectiveness and robustness of
our approach through comparing with benchmark strategies. The performance of
three representative learning strategies under our social learning framework
with our optimal rewiring is investigated as well.
DICOD: Distributed Convolutional Sparse Coding
Authors: Thomas Moreau, Laurent Oudre, Nicolas Vayatis
Date: 2017-05-29 00:00:00

In this paper, we introduce DICOD, a convolutional sparse coding algorithm
which builds shift invariant representations for long signals. This algorithm
is designed to run in a distributed setting, with local message passing, making
it communication efficient. It is based on coordinate descent and uses locally
greedy updates which accelerate the resolution compared to greedy coordinate
selection. We prove the convergence of this algorithm and highlight its
computational speed-up which is super-linear in the number of cores used. We
also provide empirical evidence for the acceleration properties of our
algorithm compared to state-of-the-art methods.
Bag-of-Words as Target for Neural Machine Translation
Authors: Shuming Ma, Xu sun, Yizhong Wang, Junyang Lin
Date: 2018-05-13 00:00:00
Proceeding: ACL 2018 7
Tasks: Machine Translation, Sentence, Translation

A sentence can be translated into more than one correct sentences. However,
most of the existing neural machine translation models only use one of the
correct translations as the targets, and the other correct sentences are
punished as the incorrect sentences in the training stage. Since most of the
correct translations for one sentence share the similar bag-of-words, it is
possible to distinguish the correct translations from the incorrect ones by the
bag-of-words. In this paper, we propose an approach that uses both the
sentences and the bag-of-words as targets in the training stage, in order to
encourage the model to generate the potentially correct sentences that are not
appeared in the training set. We evaluate our model on a Chinese-English
translation dataset, and experiments show our model outperforms the strong
baselines by the BLEU score of 4.55.
Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarization
Authors: Shuming Ma, Xu sun, Junyang Lin, Houfeng Wang
Date: 2018-05-13 00:00:00
Proceeding: ACL 2018 7
Tasks: Abstractive Text Summarization, Text Summarization

Most of the current abstractive text summarization models are based on the
sequence-to-sequence model (Seq2Seq). The source content of social media is
long and noisy, so it is difficult for Seq2Seq to learn an accurate semantic
representation. Compared with the source content, the annotated summary is
short and well written. Moreover, it shares the same meaning as the source
content. In this work, we supervise the learning of the representation of the
source content with that of the summary. In implementation, we regard a summary
autoencoder as an assistant supervisor of Seq2Seq. Following previous work, we
evaluate our model on a popular Chinese social media dataset. Experimental
results show that our model achieves the state-of-the-art performances on the
benchmark dataset.
VGGFace2: A dataset for recognising faces across pose and age
Authors: Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, Andrew Zisserman
Date: 2017-10-23 00:00:00
Tasks: Face Recognition, Face Verification, Image Retrieval

In this paper, we introduce a new large-scale face dataset named VGGFace2.
The dataset contains 3.31 million images of 9131 subjects, with an average of
362.6 images for each subject. Images are downloaded from Google Image Search
and have large variations in pose, age, illumination, ethnicity and profession
(e.g. actors, athletes, politicians). The dataset was collected with three
goals in mind: (i) to have both a large number of identities and also a large
number of images for each identity; (ii) to cover a large range of pose, age
and ethnicity; and (iii) to minimize the label noise. We describe how the
dataset was collected, in particular the automated and manual filtering stages
to ensure a high accuracy for the images of each identity. To assess face
recognition performance using the new dataset, we train ResNet-50 (with and
without Squeeze-and-Excitation blocks) Convolutional Neural Networks on
VGGFace2, on MS- Celeb-1M, and on their union, and show that training on
VGGFace2 leads to improved recognition performance over pose and age. Finally,
using the models trained on these datasets, we demonstrate state-of-the-art
performance on all the IARPA Janus face recognition benchmarks, e.g. IJB-A,
IJB-B and IJB-C, exceeding the previous state-of-the-art by a large margin.
Datasets and models are publicly available.
Covariance Pooling For Facial Expression Recognition
Authors: Dinesh Acharya, Zhiwu Huang, Danda Paudel, Luc van Gool
Date: 2018-05-13 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

Classifying facial expressions into different categories requires capturing
regional distortions of facial landmarks. We believe that second-order
statistics such as covariance is better able to capture such distortions in
regional facial fea- tures. In this work, we explore the benefits of using a
man- ifold network structure for covariance pooling to improve facial
expression recognition. In particular, we first employ such kind of manifold
networks in conjunction with tradi- tional convolutional networks for spatial
pooling within in- dividual image feature maps in an end-to-end deep learning
manner. By doing so, we are able to achieve a recognition accuracy of 58.14% on
the validation set of Static Facial Expressions in the Wild (SFEW 2.0) and
87.0% on the vali- dation set of Real-World Affective Faces (RAF) Database.
Both of these results are the best results we are aware of. Besides, we
leverage covariance pooling to capture the tem- poral evolution of per-frame
features for video-based facial expression recognition. Our reported results
demonstrate the advantage of pooling image-set features temporally by stacking
the designed manifold network of covariance pool-ing on top of convolutional
network layers.
Deceiving End-to-End Deep Learning Malware Detectors using Adversarial Examples
Authors: Felix Kreuk, Assi Barak, Shir Aviv-Reuven, Moran Baruch, Benny Pinkas, Joseph Keshet
Date: 2018-02-13 00:00:00
Tasks: Deep Learning, Image Segmentation, Malware Detection, Pose Estimation, Semantic Segmentation, speech-recognition, Speech Recognition

In recent years, deep learning has shown performance breakthroughs in many
applications, such as image detection, image segmentation, pose estimation, and
speech recognition. However, this comes with a major concern: deep networks
have been found to be vulnerable to adversarial examples. Adversarial examples
are slightly modified inputs that are intentionally designed to cause a
misclassification by the model. In the domains of images and speech, the
modifications are so small that they are not seen or heard by humans, but
nevertheless greatly affect the classification of the model.
  Deep learning models have been successfully applied to malware detection. In
this domain, generating adversarial examples is not straightforward, as small
modifications to the bytes of the file could lead to significant changes in its
functionality and validity. We introduce a novel loss function for generating
adversarial examples specifically tailored for discrete input sets, such as
executable bytes. We modify malicious binaries so that they would be detected
as benign, while preserving their original functionality, by injecting a small
sequence of bytes (payload) in the binary file. We applied this approach to an
end-to-end convolutional deep learning malware detection model and show a high
rate of detection evasion. Moreover, we show that our generated payload is
robust enough to be transferable within different locations of the same file
and across different files, and that its entropy is low and similar to that of
benign data sections.
Are All Experts Equally Good? A Study of Analyst Earnings Estimates
Authors: Amir Ban, Yishay Mansour
Date: 2018-05-13 00:00:00
Tasks: All

We investigate whether experts possess differential expertise when making
predictions. We note that this would make it possible to aggregate multiple
predictions into a result that is more accurate than their consensus average,
and that the improvement prospects grow with the amount of differentiation.
Turning this argument on its head, we show how differentiation can be measured
by how much weighted aggregation improves on simple averaging. Taking
stock-market analysts as experts in their domain, we do a retrospective study
using historical quarterly earnings forecasts and actual results for large
publicly traded companies. We use it to shed new light on the Sinha et al.
(1997) result, showing that analysts indeed possess individual expertise, but
that their differentiation is modest. On the other hand, they have significant
individual bias. Together, these enable a 20%-30% accuracy improvement over
consensus average.
Discrete Factorization Machines for Fast Feature-based Recommendation
Authors: Han Liu, Xiangnan He, Fuli Feng, Liqiang Nie, Rui Liu, Hanwang Zhang
Date: 2018-05-06 00:00:00
Tasks: Binarization, Quantization

User and item features of side information are crucial for accurate
recommendation. However, the large number of feature dimensions, e.g., usually
larger than 10^7, results in expensive storage and computational cost. This
prohibits fast recommendation especially on mobile applications where the
computational resource is very limited. In this paper, we develop a generic
feature-based recommendation model, called Discrete Factorization Machine
(DFM), for fast and accurate recommendation. DFM binarizes the real-valued
model parameters (e.g., float32) of every feature embedding into binary codes
(e.g., boolean), and thus supports efficient storage and fast user-item score
computation. To avoid the severe quantization loss of the binarization, we
propose a convergent updating rule that resolves the challenging discrete
optimization of DFM. Through extensive experiments on two real-world datasets,
we show that 1) DFM consistently outperforms state-of-the-art binarized
recommendation models, and 2) DFM shows very competitive performance compared
to its real-valued version (FM), demonstrating the minimized quantization loss.
This work is accepted by IJCAI 2018.