pwc-archive/papers-with-abstracts huggingface dataset:
Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning
Authors: Chenyang Si, Ya Jing, Wei Wang, Liang Wang, Tieniu Tan
Date: 2018-05-07 00:00:00
Proceeding: ECCV 2018 9
Tasks: Action Recognition, Graph Neural Network, Human-Object Interaction Detection, Skeleton Based Action Recognition, Spatial Reasoning, Temporal Action Localization

Skeleton-based action recognition has made great progress recently, but many
problems still remain unsolved. For example, most of the previous methods model
the representations of skeleton sequences without abundant spatial structure
information and detailed temporal dynamics features. In this paper, we propose
a novel model with spatial reasoning and temporal stack learning (SR-TSL) for
skeleton based action recognition, which consists of a spatial reasoning
network (SRN) and a temporal stack learning network (TSLN). The SRN can capture
the high-level spatial structural information within each frame by a residual
graph neural network, while the TSLN can model the detailed temporal dynamics
of skeleton sequences by a composition of multiple skip-clip LSTMs. During
training, we propose a clip-based incremental loss to optimize the model. We
perform extensive experiments on the SYSU 3D Human-Object Interaction dataset
and NTU RGB+D dataset and verify the effectiveness of each network of our
model. The comparison results illustrate that our approach achieves much better
results than state-of-the-art methods.
Accelerating Neural Transformer via an Average Attention Network
Authors: Biao Zhang, Deyi Xiong, Jinsong Su
Date: 2018-05-02 00:00:00
Proceeding: ACL 2018 7
Tasks: Decoder, Machine Translation, Translation

With parallelizable attention networks, the neural Transformer is very fast
to train. However, due to the auto-regressive architecture and self-attention
in the decoder, the decoding procedure becomes slow. To alleviate this issue,
we propose an average attention network as an alternative to the self-attention
network in the decoder of the neural Transformer. The average attention network
consists of two layers, with an average layer that models dependencies on
previous positions and a gating layer that is stacked over the average layer to
enhance the expressiveness of the proposed attention network. We apply this
network on the decoder part of the neural Transformer to replace the original
target-side self-attention model. With masking tricks and dynamic programming,
our model enables the neural Transformer to decode sentences over four times
faster than its original version with almost no loss in training time and
translation performance. We conduct a series of experiments on WMT17
translation tasks, where on 6 different language pairs, we obtain robust and
consistent speed-ups in decoding.
VH-HFCN based Parking Slot and Lane Markings Segmentation on Panoramic Surround View
Authors: Yan Wu, Tao Yang, Junqiao Zhao, Linting Guan, Wei Jiang
Date: 2018-04-19 00:00:00
Tasks: Segmentation

The automatic parking is being massively developed by car manufacturers and
providers. Until now, there are two problems with the automatic parking. First,
there is no openly-available segmentation labels of parking slot on panoramic
surround view (PSV) dataset. Second, how to detect parking slot and road
structure robustly. Therefore, in this paper, we build up a public PSV dataset.
At the same time, we proposed a highly fused convolutional network (HFCN) based
segmentation method for parking slot and lane markings based on the PSV
dataset. A surround-view image is made of four calibrated images captured from
four fisheye cameras. We collect and label more than 4,200 surround view images
for this task, which contain various illuminated scenes of different types of
parking slots. A VH-HFCN network is proposed, which adopts an HFCN as the base,
with an extra efficient VH-stage for better segmenting various markings. The
VH-stage consists of two independent linear convolution paths with vertical and
horizontal convolution kernels respectively. This modification enables the
network to robustly and precisely extract linear features. We evaluated our
model on the PSV dataset and the results showed outstanding performance in
ground markings segmentation. Based on the segmented markings, parking slots
and lanes are acquired by skeletonization, hough line transform and line
arrangement.
Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks
Authors: Craig Atkinson, Brendan McCane, Lech Szymanski, Anthony Robins
Date: 2018-02-12 00:00:00
Tasks: Generative Adversarial Network

In general, neural networks are not currently capable of learning tasks in a
sequential fashion. When a novel, unrelated task is learnt by a neural network,
it substantially forgets how to solve previously learnt tasks. One of the
original solutions to this problem is pseudo-rehearsal, which involves learning
the new task while rehearsing generated items representative of the previous
task/s. This is very effective for simple tasks. However, pseudo-rehearsal has
not yet been successfully applied to very complex tasks because in these tasks
it is difficult to generate representative items. We accomplish
pseudo-rehearsal by using a Generative Adversarial Network to generate items so
that our deep network can learn to sequentially classify the CIFAR-10, SVHN and
MNIST datasets. After training on all tasks, our network loses only 1.67%
absolute accuracy on CIFAR-10 and gains 0.24% absolute accuracy on SVHN. Our
model's performance is a substantial improvement compared to the current state
of the art solution.
Semi-orthogonal Non-negative Matrix Factorization with an Application in Text Mining
Authors: Jack Yutong Li, Ruoqing Zhu, Annie Qu, Han Ye, Zhankun Sun
Date: 2018-05-07 00:00:00
Tasks: Clustering, Dimensionality Reduction, Management

Emergency Department (ED) crowding is a worldwide issue that affects the efficiency of hospital management and the quality of patient care. This occurs when the request for an admit ward-bed to receive a patient is delayed until an admission decision is made by a doctor. To reduce the overcrowding and waiting time of ED, we build a classifier to predict the disposition of patients using manually-typed nurse notes collected during triage, thereby allowing hospital staff to begin necessary preparation beforehand. However, these triage notes involve high dimensional, noisy, and also sparse text data which makes model fitting and interpretation difficult. To address this issue, we propose the semi-orthogonal non-negative matrix factorization (SONMF) for both continuous and binary design matrices to first bi-cluster the patients and words into a reduced number of topics. The subjects can then be interpreted as a non-subtractive linear combination of orthogonal basis topic vectors. These generated topic vectors provide the hospital with a direct understanding of the cause of admission. We show that by using a transformation of basis, the classification accuracy can be further increased compared to the conventional bag-of-words model and alternative matrix factorization approaches. Through simulated data experiments, we also demonstrate that the proposed method outperforms other non-negative matrix factorization (NMF) methods in terms of factorization accuracy, rate of convergence, and degree of orthogonality.
DIRECT: Deep Discriminative Embedding for Clustering of LIGO Data
Authors: Sara Bahaadini, Vahid Noroozi, Neda Rohani, Scott Coughlin, Michael Zevin, Aggelos K. Katsaggelos
Date: 2018-05-07 00:00:00
Tasks: Clustering

In this paper, benefiting from the strong ability of deep neural network in
estimating non-linear functions, we propose a discriminative embedding function
to be used as a feature extractor for clustering tasks. The trained embedding
function transfers knowledge from the domain of a labeled set of
morphologically-distinct images, known as classes, to a new domain within which
new classes can potentially be isolated and identified. Our target application
in this paper is the Gravity Spy Project, which is an effort to characterize
transient, non-Gaussian noise present in data from the Advanced Laser
Interferometer Gravitational-wave Observatory, or LIGO. Accumulating large,
labeled sets of noise features and identifying of new classes of noise lead to
a better understanding of their origin, which makes their removal from the data
and/or detectors possible.
The State of the Art in Developing Fuzzy Ontologies: A Survey
Authors: Zahra Riahi Samani, Mehrnoush Shamsfard
Date: 2018-05-06 00:00:00
Tasks: Articles, Survey

Conceptual formalism supported by typical ontologies may not be sufficient to
represent uncertainty information which is caused due to the lack of clear cut
boundaries between concepts of a domain. Fuzzy ontologies are proposed to offer
a way to deal with this uncertainty. This paper describes the state of the art
in developing fuzzy ontologies. The survey is produced by studying about 35
works on developing fuzzy ontologies from a batch of 100 articles in the field
of fuzzy ontologies.
Clustering With Pairwise Relationships: A Generative Approach
Authors: Yen-Yun Yu, Shireen Y. Elhabian, Ross T. Whitaker
Date: 2018-05-06 00:00:00
Tasks: Clustering, Constrained Clustering, Form

Semi-supervised learning (SSL) has become important in current data analysis
applications, where the amount of unlabeled data is growing exponentially and
user input remains limited by logistics and expense. Constrained clustering, as
a subclass of SSL, makes use of user input in the form of relationships between
data points (e.g., pairs of data points belonging to the same class or
different classes) and can remarkably improve the performance of unsupervised
clustering in order to reflect user-defined knowledge of the relationships
between particular data points. Existing algorithms incorporate such user
input, heuristically, as either hard constraints or soft penalties, which are
separate from any generative or statistical aspect of the clustering model;
this results in formulations that are suboptimal and not sufficiently general.
In this paper, we propose a principled, generative approach to
probabilistically model, without ad hoc penalties, the joint distribution given
by user-defined pairwise relations. The proposed model accounts for general
underlying distributions without assuming a specific form and relies on
expectation-maximization for model fitting. For distributions in a standard
form, the proposed approach results in a closed-form solution for updated
parameters.
DocFace: Matching ID Document Photos to Selfies
Authors: Yichun Shi, Anil K. Jain
Date: 2018-05-06 00:00:00
Tasks: Face Recognition, TAR, Transfer Learning

Numerous activities in our daily life, including transactions, access to
services and transportation, require us to verify who we are by showing our ID
documents containing face images, e.g. passports and driver licenses. An
automatic system for matching ID document photos to live face images in real
time with high accuracy would speedup the verification process and remove the
burden on human operators. In this paper, by employing the transfer learning
technique, we propose a new method, DocFace, to train a domain-specific network
for ID document photo matching without a large dataset. Compared with the
baseline of applying existing methods for general face recognition to this
problem, our method achieves considerable improvement. A cross validation on an
ID-Selfie dataset shows that DocFace improves the TAR from 61.14% to 92.77% at
FAR=0.1%. Experimental results also indicate that given more training data, a
viable system for automatic ID document photo matching can be developed and
deployed.
Multi-Domain Neural Machine Translation
Authors: Sander Tars, Mark Fishel
Date: 2018-05-06 00:00:00
Tasks: Machine Translation, NMT, Translation

We present an approach to neural machine translation (NMT) that supports
multiple domains in a single model and allows switching between the domains
when translating. The core idea is to treat text domains as distinct languages
and use multilingual NMT methods to create multi-domain translation systems, we
show that this approach results in significant translation quality gains over
fine-tuning. We also explore whether the knowledge of pre-specified text
domains is necessary, turns out that it is after all, but also that when it is
not known quite high translation quality can be reached.
SqueezeJet: High-level Synthesis Accelerator Design for Deep Convolutional Neural Networks
Authors: Panagiotis G. Mousouliotis, Loukas P. Petrou
Date: 2018-05-06 00:00:00
Tasks: High-Level Synthesis, object-detection, Object Detection, Object Recognition, Vocal Bursts Intensity Prediction

Deep convolutional neural networks have dominated the pattern recognition
scene by providing much more accurate solutions in computer vision problems
such as object recognition and object detection. Most of these solutions come
at a huge computational cost, requiring billions of multiply-accumulate
operations and, thus, making their use quite challenging in real-time
applications that run on embedded mobile (resource-power constrained) hardware.
This work presents the architecture, the high-level synthesis design, and the
implementation of SqueezeJet, an FPGA accelerator for the inference phase of
the SqueezeNet DCNN architecture, which is designed specifically for use in
embedded systems. Results show that SqueezeJet can achieve 15.16 times speed-up
compared to the software implementation of SqueezeNet running on an embedded
mobile processor with less than 1% drop in top-5 accuracy.
Dropping Networks for Transfer Learning
Authors: James O' Neill, Danushka Bollegala
Date: 2018-04-23 00:00:00
Tasks: Few-Shot Learning, Natural Language Inference, Natural Language Understanding, Transfer Learning

Many tasks in natural language understanding require learning relationships
between two sequences for various tasks such as natural language inference,
paraphrasing and entailment. These aforementioned tasks are similar in nature,
yet they are often modeled individually. Knowledge transfer can be effective
for closely related tasks. However, transferring all knowledge, some of which
irrelevant for a target task, can lead to sub-optimal results due to
\textit{negative} transfer. Hence, this paper focuses on the transferability of
both instances and parameters across natural language understanding tasks by
proposing an ensemble-based transfer learning method. \newline The primary
contribution of this paper is the combination of both \textit{Dropout} and
\textit{Bagging} for improved transferability in neural networks, referred to
as \textit{Dropping} herein. We present a straightforward yet novel approach
for incorporating source \textit{Dropping} Networks to a target task for
few-shot learning that mitigates \textit{negative} transfer. This is achieved
by using a decaying parameter chosen according to the slope changes of a
smoothed spline error curve at sub-intervals during training. We compare the
proposed approach against hard parameter sharing and soft parameter sharing
transfer methods in the few-shot learning case. We also compare against models
that are fully trained on the target task in the standard supervised learning
setup. The aforementioned adjustment leads to improved transfer learning
performance and comparable results to the current state of the art only using a
fraction of the data from the target task.
Coherence Modeling of Asynchronous Conversations: A Neural Entity Grid Approach
Authors: Tasnim Mohiuddin, Shafiq Joty, Dat Tien Nguyen
Date: 2018-05-06 00:00:00
Proceeding: ACL 2018 7

We propose a novel coherence model for written asynchronous conversations
(e.g., forums, emails), and show its applications in coherence assessment and
thread reconstruction tasks. We conduct our research in two steps. First, we
propose improvements to the recently proposed neural entity grid model by
lexicalizing its entity transitions. Then, we extend the model to asynchronous
conversations by incorporating the underlying conversational structure in the
entity grid representation and feature computation. Our model achieves state of
the art results on standard coherence assessment tasks in monologue and
conversations outperforming existing models. We also demonstrate its
effectiveness in reconstructing thread structures.
Limited Evaluation Cooperative Co-evolutionary Differential Evolution for Large-scale Neuroevolution
Authors: Anil Yaman, Decebal Constantin Mocanu, Giovanni Iacca, George Fletcher, Mykola Pechenizkiy
Date: 2018-04-19 00:00:00
Tasks: Evolutionary Algorithms

Many real-world control and classification tasks involve a large number of
features. When artificial neural networks (ANNs) are used for modeling these
tasks, the network architectures tend to be large. Neuroevolution is an
effective approach for optimizing ANNs; however, there are two bottlenecks that
make their application challenging in case of high-dimensional networks using
direct encoding. First, classic evolutionary algorithms tend not to scale well
for searching large parameter spaces; second, the network evaluation over a
large number of training instances is in general time-consuming. In this work,
we propose an approach called the Limited Evaluation Cooperative
Co-evolutionary Differential Evolution algorithm (LECCDE) to optimize
high-dimensional ANNs.
  The proposed method aims to optimize the pre-synaptic weights of each
post-synaptic neuron in different subpopulations using a Cooperative
Co-evolutionary Differential Evolution algorithm, and employs a limited
evaluation scheme where fitness evaluation is performed on a relatively small
number of training instances based on fitness inheritance. We test LECCDE on
three datasets with various sizes, and our results show that cooperative
co-evolution significantly improves the test error comparing to standard
Differential Evolution, while the limited evaluation scheme facilitates a
significant reduction in computing time.
Deep density networks and uncertainty in recommender systems
Authors: Yoel Zeldes, Stavros Theodorakis, Efrat Solodnik, Aviv Rotman, Gil Chamiel, Dan Friedman
Date: 2017-11-07 00:00:00
Proceeding: ICLR 2018 1
Tasks: Collaborative Filtering, Efficient Exploration, Recommendation Systems

Building robust online content recommendation systems requires learning
complex interactions between user preferences and content features. The field
has evolved rapidly in recent years from traditional multi-arm bandit and
collaborative filtering techniques, with new methods employing Deep Learning
models to capture non-linearities. Despite progress, the dynamic nature of
online recommendations still poses great challenges, such as finding the
delicate balance between exploration and exploitation. In this paper we show
how uncertainty estimations can be incorporated by employing them in an
optimistic exploitation/exploration strategy for more efficient exploration of
new recommendations. We provide a novel hybrid deep neural network model, Deep
Density Networks (DDN), which integrates content-based deep learning models
with a collaborative scheme that is able to robustly model and estimate
uncertainty. Finally, we present online and offline results after incorporating
DNN into a real world content recommendation system that serves billions of
recommendations per day, and show the benefit of using DDN in practice.
Breaking NLI Systems with Sentences that Require Simple Lexical Inferences
Authors: Max Glockner, Vered Shwartz, Yoav Goldberg
Date: 2018-05-06 00:00:00
Proceeding: ACL 2018 7
Tasks: World Knowledge

We create a new NLI test set that shows the deficiency of state-of-the-art
models in inferences that require lexical and world knowledge. The new examples
are simpler than the SNLI test set, containing sentences that differ by at most
one word from sentences in the training set. Yet, the performance on the new
test set is substantially worse across systems trained on SNLI, demonstrating
that these systems are limited in their generalization ability, failing to
capture many simple inferences.
Automated Diagnosis of Clinic Workflows
Authors: Alex Cheng, Jules White
Date: 2018-05-06 00:00:00

Outpatient clinics often run behind schedule due to patients who arrive late
or appointments that run longer than expected. We sought to develop a
generalizable method that would allow healthcare providers to diagnose problems
in workflow that disrupt the schedule on any given provider clinic day. We use
a constraint optimization problem to identify the least number of appointment
modifications that make the rest of the schedule run on-time. We apply this
method to an outpatient clinic at Vanderbilt. For patient seen in this clinic
between March 27, 2017 and April 21, 2017, long cycle times tended to affect
the overall schedule more than late patients. Results from this workflow
diagnosis method could be used to inform interventions to help clinics run
smoothly, thus decreasing patient wait times and increasing provider
utilization.
Construction of the Literature Graph in Semantic Scholar
Authors: Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, Oren Etzioni
Date: 2018-05-06 00:00:00
Proceeding: NAACL 2018 6
Tasks: Entity Extraction using GAN, graph construction

We describe a deployed scalable system for organizing published scientific
literature into a heterogeneous graph to facilitate algorithmic manipulation
and discovery. The resulting literature graph consists of more than 280M nodes,
representing papers, authors, entities and various interactions between them
(e.g., authorships, citations, entity mentions). We reduce literature graph
construction into familiar NLP tasks (e.g., entity extraction and linking),
point out research challenges due to differences from standard formulations of
these tasks, and report empirical results for each task. The methods described
in this paper are used to enable semantic features in www.semanticscholar.org
Russian word sense induction by clustering averaged word embeddings
Authors: Andrey Kutuzov
Date: 2018-05-06 00:00:00
Tasks: Clustering, Word Embeddings, Word Sense Induction

The paper reports our participation in the shared task on word sense
induction and disambiguation for the Russian language (RUSSE-2018). Our team
was ranked 2nd for the wiki-wiki dataset (containing mostly homonyms) and 5th
for the bts-rnc and active-dict datasets (containing mostly polysemous words)
among all 19 participants.
  The method we employed was extremely naive. It implied representing contexts
of ambiguous words as averaged word embedding vectors, using off-the-shelf
pre-trained distributional models. Then, these vector representations were
clustered with mainstream clustering techniques, thus producing the groups
corresponding to the ambiguous word senses. As a side result, we show that word
embedding models trained on small but balanced corpora can be superior to those
trained on large but noisy data - not only in intrinsic evaluation, but also in
downstream tasks like word sense induction.
Image Based Fashion Product Recommendation with Deep Learning
Authors: Hessel Tuinhof, Clemens Pirker, Markus Haltmeier
Date: 2018-05-06 00:00:00
Tasks: Deep Learning, Product Recommendation, Recommendation Systems, Transfer Learning

We develop a two-stage deep learning framework that recommends fashion images
based on other input images of similar style. For that purpose, a neural
network classifier is used as a data-driven, visually-aware feature extractor.
The latter then serves as input for similarity-based recommendations using a
ranking algorithm. Our approach is tested on the publicly available Fashion
dataset. Initialization strategies using transfer learning from larger product
databases are presented. Combined with more traditional content-based
recommendation systems, our framework can help to increase robustness and
performance, for example, by better matching a particular customer style.