pwc-archive/papers-with-abstracts huggingface dataset:
Memory Efficient Experience Replay for Streaming Learning
Authors: Tyler L. Hayes, Nathan D. Cahill, Christopher Kanan
Date: 2018-09-16 00:00:00

In supervised machine learning, an agent is typically trained once and then
deployed. While this works well for static settings, robots often operate in
changing environments and must quickly learn new things from data streams. In
this paradigm, known as streaming learning, a learner is trained online, in a
single pass, from a data stream that cannot be assumed to be independent and
identically distributed (iid). Streaming learning will cause conventional deep
neural networks (DNNs) to fail for two reasons: 1) they need multiple passes
through the entire dataset; and 2) non-iid data will cause catastrophic
forgetting. An old fix to both of these issues is rehearsal. To learn a new
example, rehearsal mixes it with previous examples, and then this mixture is
used to update the DNN. Full rehearsal is slow and memory intensive because it
stores all previously observed examples, and its effectiveness for preventing
catastrophic forgetting has not been studied in modern DNNs. Here, we describe
the ExStream algorithm for memory efficient rehearsal and compare it to
alternatives. We find that full rehearsal can eliminate catastrophic forgetting
in a variety of streaming learning settings, with ExStream performing well
using far less memory and computation.
Curriculum-Based Neighborhood Sampling For Sequence Prediction
Authors: James O' Neill, Danushka Bollegala
Date: 2018-09-16 00:00:00
Tasks: Language Modeling, Language Modelling, Prediction

The task of multi-step ahead prediction in language models is challenging
considering the discrepancy between training and testing. At test time, a
language model is required to make predictions given past predictions as input,
instead of the past targets that are provided during training. This difference,
known as exposure bias, can lead to the compounding of errors along a generated
sequence at test time.
  In order to improve generalization in neural language models and address
compounding errors, we propose a curriculum learning based method that
gradually changes an initially deterministic teacher policy to a gradually more
stochastic policy, which we refer to as \textit{Nearest-Neighbor Replacement
Sampling}. A chosen input at a given timestep is replaced with a sampled
nearest neighbor of the past target with a truncated probability proportional
to the cosine similarity between the original word and its top $k$ most similar
words. This allows the teacher to explore alternatives when the teacher
provides a sub-optimal policy or when the initial policy is difficult for the
learner to model. The proposed strategy is straightforward, online and requires
little additional memory requirements. We report our main findings on two
language modelling benchmarks and find that the proposed approach performs
particularly well when used in conjunction with scheduled sampling, that too
attempts to mitigate compounding errors in language models.
MeshCNN: A Network with an Edge
Authors: Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, Daniel Cohen-Or
Date: 2018-09-16 00:00:00
Tasks: 3D Object Classification, 3D Part Segmentation, Cube Engraving Classification

Polygonal meshes provide an efficient representation for 3D shapes. They
explicitly capture both shape surface and topology, and leverage non-uniformity
to represent large flat regions as well as sharp, intricate features. This
non-uniformity and irregularity, however, inhibits mesh analysis efforts using
neural networks that combine convolution and pooling operations. In this paper,
we utilize the unique properties of the mesh for a direct analysis of 3D shapes
using MeshCNN, a convolutional neural network designed specifically for
triangular meshes. Analogous to classic CNNs, MeshCNN combines specialized
convolution and pooling layers that operate on the mesh edges, by leveraging
their intrinsic geodesic connections. Convolutions are applied on edges and the
four edges of their incident triangles, and pooling is applied via an edge
collapse operation that retains surface topology, thereby, generating new mesh
connectivity for the subsequent convolutions. MeshCNN learns which edges to
collapse, thus forming a task-driven process where the network exposes and
expands the important features while discarding the redundant ones. We
demonstrate the effectiveness of our task-driven pooling on various learning
tasks applied to 3D meshes.
A Storm in an IoT Cup: The Emergence of Cyber-Physical Social Machines
Authors: Aastha Madaan, Jason R. C. Nurse, David De Roure, Kieron O'Hara, Wendy Hall, Sadie Creese
Date: 2018-09-16 00:00:00

The concept of social machines is increasingly being used to characterise
various socio-cognitive spaces on the Web. Social machines are human
collectives using networked digital technology which initiate real-world
processes and activities including human communication, interactions and
knowledge creation. As such, they continuously emerge and fade on the Web. The
relationship between humans and machines is made more complex by the adoption
of Internet of Things (IoT) sensors and devices. The scale, automation,
continuous sensing, and actuation capabilities of these devices add an extra
dimension to the relationship between humans and machines making it difficult
to understand their evolution at either the systemic or the conceptual level.
This article describes these new socio-technical systems, which we term
Cyber-Physical Social Machines, through different exemplars, and considers the
associated challenges of security and privacy.
Classifying Process Instances Using Recurrent Neural Networks
Authors: Markku Hinkka, Teemu Lehto, Keijo Heljanko, Alexander Jung
Date: 2018-09-16 00:00:00
Tasks: General Classification

Process Mining consists of techniques where logs created by operative systems
are transformed into process models. In process mining tools it is often
desired to be able to classify ongoing process instances, e.g., to predict how
long the process will still require to complete, or to classify process
instances to different classes based only on the activities that have occurred
in the process instance thus far. Recurrent neural networks and its subclasses,
such as Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM), have been
demonstrated to be able to learn relevant temporal features for subsequent
classification tasks. In this paper we apply recurrent neural networks to
classifying process instances. The proposed model is trained in a supervised
fashion using labeled process instances extracted from event log traces. This
is the first time we know of GRU having been used in classifying business
process instances. Our main experimental results shows that GRU outperforms
LSTM remarkably in training time while giving almost identical accuracies to
LSTM models. Additional contributions of our paper are improving the
classification model training time by filtering infrequent activities, which is
a technique commonly used, e.g., in Natural Language Processing (NLP).
Semantic Interoperability Middleware Architecture for Heterogeneous Environmental Data Sources
Authors: A. K. Akanbi, M. Masinde
Date: 2018-09-16 00:00:00

Data heterogeneity hampers the effort to integrate and infer knowledge from
vast heterogeneous data sources. An application case study is described, in
which the objective was to semantically represent and integrate structured data
from sensor devices with unstructured data in the form of local indigenous
knowledge. However, the semantic representation of these heterogeneous data
sources for environmental monitoring systems is not well supported yet. To
combat the incompatibility issues, a dedicated semantic middleware solution is
required. In this paper, we describe and evaluate a cross-domain middleware
architecture that semantically integrates and generate inference from
heterogeneous data sources. These use of semantic technology for predicting and
forecasting complex environmental phenomenon will increase the degree of
accuracy of environmental monitoring systems.
Comparison of Deep Learning and the Classical Machine Learning Algorithm for the Malware Detection
Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore
Date: 2018-09-16 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, General Classification, Language Modeling, Language Modelling, Machine Translation, Malware Classification, Malware Detection, Translation

Recently, Deep Learning has been showing promising results in various
Artificial Intelligence applications like image recognition, natural language
processing, language modeling, neural machine translation, etc. Although, in
general, it is computationally more expensive as compared to classical machine
learning techniques, their results are found to be more effective in some
cases. Therefore, in this paper, we investigated and compared one of the Deep
Learning Architecture called Deep Neural Network (DNN) with the classical
Random Forest (RF) machine learning algorithm for the malware classification.
We studied the performance of the classical RF and DNN with 2, 4 & 7 layers
architectures with the four different feature sets, and found that irrespective
of the features inputs, the classical RF accuracy outperforms the DNN.
An investigation of a deep learning based malware detection system
Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore
Date: 2018-09-16 00:00:00
Tasks: Deep Learning, Feature Engineering, Malware Detection

We investigate a Deep Learning based system for malware detection. In the
investigation, we experiment with different combination of Deep Learning
architectures including Auto-Encoders, and Deep Neural Networks with varying
layers over Malicia malware dataset on which earlier studies have obtained an
accuracy of (98%) with an acceptable False Positive Rates (1.07%). But these
results were done using extensive man-made custom domain features and investing
corresponding feature engineering and design efforts. In our proposed approach,
besides improving the previous best results (99.21% accuracy and a False
Positive Rate of 0.19%) indicates that Deep Learning based systems could
deliver an effective defense against malware. Since it is good in automatically
extracting higher conceptual features from the data, Deep Learning based
systems could provide an effective, general and scalable mechanism for
detection of existing and unknown malware.
Meta-Embedding as Auxiliary Task Regularization
Authors: James O' Neill, Danushka Bollegala
Date: 2018-09-16 00:00:00
Tasks: Self-Supervised Learning, Sentence, Sentence Classification, Word Embeddings, Word Similarity

Word embeddings have been shown to benefit from ensambling several word embedding sources, often carried out using straightforward mathematical operations over the set of word vectors. More recently, self-supervised learning has been used to find a lower-dimensional representation, similar in size to the individual word embeddings within the ensemble. However, these methods do not use the available manually labeled datasets that are often used solely for the purpose of evaluation. We propose to reconstruct an ensemble of word embeddings as an auxiliary task that regularises a main task while both tasks share the learned meta-embedding layer. We carry out intrinsic evaluation (6 word similarity datasets and 3 analogy datasets) and extrinsic evaluation (4 downstream tasks). For intrinsic task evaluation, supervision comes from various labeled word similarity datasets. Our experimental results show that the performance is improved for all word similarity datasets when compared to self-supervised learning methods with a mean increase of $11.33$ in Spearman correlation. Specifically, the proposed method shows the best performance in 4 out of 6 of word similarity datasets when using a cosine reconstruction loss and Brier's word similarity loss. Moreover, improvements are also made when performing word meta-embedding reconstruction in sequence tagging and sentence meta-embedding for sentence classification.
An FPGA-Accelerated Design for Deep Learning Pedestrian Detection in Self-Driving Vehicles
Authors: Abdallah Moussawi, Kamal Haddad, Anthony Chahine
Date: 2018-09-16 00:00:00
Tasks: object-detection, Object Detection, Pedestrian Detection

With the rise of self-driving vehicles comes the risk of accidents and the
need for higher safety, and protection for pedestrian detection in the
following scenarios: imminent crashes, thus the car should crash into an object
and avoid the pedestrian, and in the case of road intersections, where it is
important for the car to stop when pedestrians are crossing. Currently, a
special topology of deep neural networks called Fused Deep Neural Network
(F-DNN) is considered to be the state of the art in pedestrian detection, as it
has the lowest miss rate, yet it is very slow. Therefore, acceleration is
needed to speed up the performance. This project proposes two contributions to
address this problem, by using a deep neural network used for object detection,
called Single Shot Multi-Box Detector (SSD). The first contribution is training
and tuning the hyperparameters of SSD to improve pedestrian detection. The
second contribution is a new FPGA design for accelerating the model on the
Altera Arria 10 platform. The final system will be used in self-driving
vehicles in real-time. Preliminary results of the improved SSD shows 3% higher
miss-rate than F-DNN on Caltech pedestrian detection benchmark, but 4x
performance improvement. The acceleration design is expected to achieve an
additional performance improvement significantly outweighing the minimal
difference in accuracy.
Road Detection Technique Using Filters with Application to Autonomous Driving System
Authors: Y. O. Agunbiade, J. O. Dehinbo, T. Zuva, A. K. Akanbi
Date: 2018-09-16 00:00:00
Tasks: Autonomous Driving, General Classification

Autonomous driving systems are broadly used equipment in the industries and
in our daily lives, they assist in production, but are majorly used for
exploration in dangerous or unfamiliar locations. Thus, for a successful
exploration, navigation plays a significant role. Road detection is an
essential factor that assists autonomous robots achieved perfect navigation.
Various techniques using camera sensors have been proposed by numerous scholars
with inspiring results, but their techniques are still vulnerable to these
environmental noises: rain, snow, light intensity and shadow. In addressing
these problems, this paper proposed to enhance the road detection system with
filtering algorithm to overcome these limitations. Normalized Differences Index
(NDI) and morphological operation are the filtering algorithms used to address
the effect of shadow and guidance and re-guidance image filtering algorithms
are used to address the effect of rain and/or snow, while dark channel image
and specular-to-diffuse are the filters used to address light intensity
effects. The experimental performance of the road detection system with
filtering algorithms was tested qualitatively and quantitatively using the
following evaluation schemes: False Negative Rate (FNR) and False Positive Rate
(FPR). Comparison results of the road detection system with and without
filtering algorithm shows the filtering algorithm's capability to suppress the
effect of environmental noises because better road/non-road classification is
achieved by the road detection system. with filtering algorithm. This
achievement has further improved path planning/region classification for
autonomous driving system
A Data Analytics Framework for Aggregate Data Analysis
Authors: Sanket Tavarageri, Nag Mani, Anand Ramasubramanian, Jaskiran Kalsi
Date: 2018-09-16 00:00:00
Tasks: BIG-bench Machine Learning, Marketing, valid

In many contexts, we have access to aggregate data, but individual level data
is unavailable. For example, medical studies sometimes report only aggregate
statistics about disease prevalence because of privacy concerns. Even so, many
a time it is desirable, and in fact could be necessary to infer individual
level characteristics from aggregate data. For instance, other researchers who
want to perform more detailed analysis of disease characteristics would require
individual level data. Similar challenges arise in other fields too including
politics, and marketing.
  In this paper, we present an end-to-end pipeline for processing of aggregate
data to derive individual level statistics, and then using the inferred data to
train machine learning models to answer questions of interest. We describe a
novel algorithm for reconstructing fine-grained data from summary statistics.
This step will create multiple candidate datasets which will form the input to
the machine learning models. The advantage of the highly parallel architecture
we propose is that uncertainty in the generated fine-grained data will be
compensated by the use of multiple candidate fine-grained datasets.
Consequently, the answers derived from the machine learning models will be more
valid and usable. We validate our approach using data from a challenging
medical problem called Acute Traumatic Coagulopathy.
Inspiration Learning through Preferences
Authors: Nir Baram, Shie Mannor
Date: 2018-09-16 00:00:00
Tasks: Imitation Learning, Reinforcement Learning, Transfer Learning

Current imitation learning techniques are too restrictive because they
require the agent and expert to share the same action space. However,
oftentimes agents that act differently from the expert can solve the task just
as good. For example, a person lifting a box can be imitated by a ceiling
mounted robot or a desktop-based robotic-arm. In both cases, the end goal of
lifting the box is achieved, perhaps using different strategies. We denote this
setup as \textit{Inspiration Learning} - knowledge transfer between agents that
operate in different action spaces. Since state-action expert demonstrations
can no longer be used, Inspiration learning requires novel methods to guide the
agent towards the end goal. In this work, we rely on ideas of Preferential
based Reinforcement Learning (PbRL) to design Advantage Actor-Critic algorithms
for solving inspiration learning tasks. Unlike classic actor-critic
architectures, the critic we use consists of two parts: a) a state-value
estimation as in common actor-critic algorithms and b) a single step reward
function derived from an expert/agent classifier. We show that our method is
capable of extending the current imitation framework to new horizons. This
includes continuous-to-discrete action imitation, as well as primitive-to-macro
action imitation.
On-Line Learning of Linear Dynamical Systems: Exponential Forgetting in Kalman Filters
Authors: Mark Kozdoba, Jakub Marecek, Tigran Tchrakian, Shie Mannor
Date: 2018-09-16 00:00:00
Proceeding: AAAI 2019
Tasks: regression, Time Series, Time Series Analysis, Time Series Forecasting

Kalman filter is a key tool for time-series forecasting and analysis. We show
that the dependence of a prediction of Kalman filter on the past is decaying
exponentially, whenever the process noise is non-degenerate. Therefore, Kalman
filter may be approximated by regression on a few recent observations.
Surprisingly, we also show that having some process noise is essential for the
exponential decay. With no process noise, it may happen that the forecast
depends on all of the past uniformly, which makes forecasting more difficult.
  Based on this insight, we devise an on-line algorithm for improper learning
of a linear dynamical system (LDS), which considers only a few most recent
observations. We use our decay results to provide the first regret bounds
w.r.t. to Kalman filters within learning an LDS. That is, we compare the
results of our algorithm to the best, in hindsight, Kalman filter for a given
signal. Also, the algorithm is practical: its per-update run-time is linear in
the regression depth.
In Defense of the Classification Loss for Person Re-Identification
Authors: Yao Zhai, Xun Guo, Yan Lu, Houqiang Li
Date: 2018-09-16 00:00:00
Tasks: Classification, General Classification, Metric Learning, Person Re-Identification, Triplet

The recent research for person re-identification has been focused on two
trends. One is learning the part-based local features to form more informative
feature descriptors. The other is designing effective metric learning loss
functions such as the triplet loss family. We argue that learning global
features with classification loss could achieve the same goal, even with some
simple and cost-effective architecture design. In this paper, we first explain
why the person re-id framework with standard classification loss usually has
inferior performance compared to metric learning. Based on that, we further
propose a person re-id framework featured by channel grouping and multi-branch
strategy, which divides global features into multiple channel groups and learns
the discriminative channel group features by multi-branch classification
layers. The extensive experiments show that our framework outperforms prior
state-of-the-arts in terms of both accuracy and inference speed.
f-VAEs: Improve VAEs with Conditional Flows
Authors: Jianlin Su, Guang Wu
Date: 2018-09-16 00:00:00

In this paper, we integrate VAEs and flow-based generative models
successfully and get f-VAEs. Compared with VAEs, f-VAEs generate more vivid
images, solved the blurred-image problem of VAEs. Compared with flow-based
models such as Glow, f-VAE is more lightweight and converges faster, achieving
the same performance under smaller-size architecture.
Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
Authors: Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, Kun Zhang, DaCheng Tao
Date: 2018-09-16 00:00:00
Proceeding: CVPR 2019 6
Tasks: Generative Adversarial Network

Unsupervised domain mapping aims to learn a function to translate domain X to
Y by a function GXY in the absence of paired examples. Finding the optimal GXY
without paired data is an ill-posed problem, so appropriate constraints are
required to obtain reasonable solutions. One of the most prominent constraints
is cycle consistency, which enforces the translated image by GXY to be
translated back to the input image by an inverse mapping GYX. While cycle
consistency requires the simultaneous training of GXY and GY X, recent studies
have shown that one-sided domain mapping can be achieved by preserving pairwise
distances between images. Although cycle consistency and distance preservation
successfully constrain the solution space, they overlook the special properties
that simple geometric transformations do not change the semantic structure of
images. Based on this special property, we develop a geometry-consistent
generative adversarial network (GcGAN), which enables one-sided unsupervised
domain mapping. GcGAN takes the original image and its counterpart image
transformed by a predefined geometric transformation as inputs and generates
two images in the new domain coupled with the corresponding
geometry-consistency constraint. The geometry-consistency constraint reduces
the space of possible solutions while keep the correct solutions in the search
space. Quantitative and qualitative comparisons with the baseline (GAN alone)
and the state-of-the-art methods including CycleGAN and DistanceGAN demonstrate
the effectiveness of our method.