pwc-archive/papers-with-abstracts huggingface dataset:
Saliency-Guided Attention Network for Image-Sentence Matching
Authors: Zhong Ji, Haoran Wang, Jungong Han, Yanwei Pang
Date: 2019-04-20 00:00:00
Proceeding: ICCV 2019 10
Tasks: Sentence

This paper studies the task of matching image and sentence, where learning appropriate representations across the multi-modal data appears to be the main challenge. Unlike previous approaches that predominantly deploy symmetrical architecture to represent both modalities, we propose Saliency-guided Attention Network (SAN) that asymmetrically employs visual and textual attention modules to learn the fine-grained correlation intertwined between vision and language. The proposed SAN mainly includes three components: saliency detector, Saliency-weighted Visual Attention (SVA) module, and Saliency-guided Textual Attention (STA) module. Concretely, the saliency detector provides the visual saliency information as the guidance for the two attention modules. SVA is designed to leverage the advantage of the saliency information to improve discrimination of visual representations. By fusing the visual information from SVA and textual information as a multi-modal guidance, STA learns discriminative textual representations that are highly sensitive to visual clues. Extensive experiments demonstrate SAN can substantially improve the state-of-the-art results on the benchmark Flickr30K and MSCOCO datasets by a large margin.
Facial Feature Embedded CycleGAN for VIS-NIR Translation
Authors: Huijiao Wang, Li Wang, Xulei Yang, Lei Yu, Haijian Zhang
Date: 2019-04-20 00:00:00
Tasks: Face Recognition, Translation

VIS-NIR face recognition remains a challenging task due to the distinction
between spectral components of two modalities and insufficient paired training
data. Inspired by the CycleGAN, this paper presents a method aiming to
translate VIS face images into fake NIR images whose distributions are intended
to approximate those of true NIR images, which is achieved by proposing a new
facial feature embedded CycleGAN. Firstly, to learn the particular feature of
NIR domain while preserving common facial representation between VIS and NIR
domains, we employ a general facial feature extractor (FFE) to replace the
encoder in the original generator of CycleGAN. For implementing the facial
feature extractor, herein the MobileFaceNet is pretrained on a VIS face
database, and is able to extract effective features. Secondly, the
domain-invariant feature learning is enhanced by considering a new pixel
consistency loss. Lastly, we establish a new WHU VIS-NIR database which varies
in face rotation and expressions to enrich the training data. Experimental
results on the Oulu-CASIA NIR-VIS database and the WHU VIS-NIR database show
that the proposed FFE-based CycleGAN (FFE-CycleGAN) outperforms
state-of-the-art VIS-NIR face recognition methods and achieves 96.5\% accuracy.
A Differential Approach for Gaze Estimation
Authors: Gang Liu, Yu Yu, Kenneth A. Funes Mora, Jean-Marc Odobez
Date: 2019-04-20 00:00:00
Tasks: Gaze Estimation

Non-invasive gaze estimation methods usually regress gaze directions directly from a single face or eye image. However, due to important variabilities in eye shapes and inner eye structures amongst individuals, universal models obtain limited accuracies and their output usually exhibit high variance as well as biases which are subject dependent. Therefore, increasing accuracy is usually done through calibration, allowing gaze predictions for a subject to be mapped to his/her actual gaze. In this paper, we introduce a novel image differential method for gaze estimation. We propose to directly train a differential convolutional neural network to predict the gaze differences between two eye input images of the same subject. Then, given a set of subject specific calibration images, we can use the inferred differences to predict the gaze direction of a novel eye sample. The assumption is that by allowing the comparison between two eye images, annoyance factors (alignment, eyelid closing, illumination perturbations) which usually plague single image prediction methods can be much reduced, allowing better prediction altogether. Experiments on 3 public datasets validate our approach which constantly outperforms state-of-the-art methods even when using only one calibration sample or when the latter methods are followed by subject specific gaze adaptation.
An Unsupervised Joint System for Text Generation from Knowledge Graphs and Semantic Parsing
Authors: Martin Schmitt, Sahand Sharifzadeh, Volker Tresp, Hinrich Schütze
Date: 2019-04-20 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Domain Adaptation, Unsupervised KG-to-Text Generation, Unsupervised semantic parsing

Knowledge graphs (KGs) can vary greatly from one domain to another. Therefore supervised approaches to both graph-to-text generation and text-to-graph knowledge extraction (semantic parsing) will always suffer from a shortage of domain-specific parallel graph-text data; at the same time, adapting a model trained on a different domain is often impossible due to little or no overlap in entities and relations. This situation calls for an approach that (1) does not need large amounts of annotated data and thus (2) does not need to rely on domain adaptation techniques to work well in different domains. To this end, we present the first approach to unsupervised text generation from KGs and show simultaneously how it can be used for unsupervised semantic parsing. We evaluate our approach on WebNLG v2.1 and a new benchmark leveraging scene graphs from Visual Genome. Our system outperforms strong baselines for both text$\leftrightarrow$graph conversion tasks without any manual adaptation from one dataset to the other. In additional experiments, we investigate the impact of using different unsupervised objectives.
Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings
Authors: Haozhou Wang, James Henderson, Paola Merlo
Date: 2019-04-20 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Cross-Lingual Word Embeddings, Sentence, Word Embeddings

Distributed representations of words which map each word to a continuous
vector have proven useful in capturing important linguistic information not
only in a single language but also across different languages. Current
unsupervised adversarial approaches show that it is possible to build a mapping
matrix that align two sets of monolingual word embeddings together without high
quality parallel data such as a dictionary or a sentence-aligned corpus.
However, without post refinement, the performance of these methods' preliminary
mapping is not good, leading to poor performance for typologically distant
languages.
  In this paper, we propose a weakly-supervised adversarial training method to
overcome this limitation, based on the intuition that mapping across languages
is better done at the concept level than at the word level. We propose a
concept-based adversarial training method which for most languages improves the
performance of previous unsupervised adversarial methods, especially for
typologically distant language pairs.
Learning the Right Expansion-ordering Heuristics for Satisfiability Testing in OWL Reasoners
Authors: Razieh Mehri, Volker Haarslev, Hamidreza Chinaei
Date: 2019-04-20 00:00:00

Web Ontology Language (OWL) reasoners are used to infer new logical relations
from ontologies. While inferring new facts, these reasoners can be further
optimized, e.g., by properly ordering disjuncts in disjunction expressions of
ontologies for satisfiability testing of concepts. Different expansion-ordering
heuristics have been developed for this purpose. The built-in heuristics in
these reasoners determine the order for branches in search trees while each
heuristic choice causes different effects for various ontologies depending on
the ontologies' syntactic structure and probably other features as well. A
learning-based approach that takes into account the features aims to select an
appropriate expansion-ordering heuristic for each ontology. The proper choice
is expected to accelerate the reasoning process for the reasoners. In this
paper, the effect of our methodology is investigated on a well-known reasoner
that is JFact. Our experiments show the average speedup by a factor of one to
two orders of magnitude for satisfiability testing after applying learning
methodology for selecting the right expansion-ordering heuristics.
Personalized sentence generation using generative adversarial networks with author-specific word usage
Authors: Chenhan Yuan, Yi-chin Huang
Date: 2019-04-20 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Sentence

The author-specific word usage is a vital feature to let readers perceive the
writing style of the author. In this work, a personalized sentence generation
method based on generative adversarial networks (GANs) is proposed to cope with
this issue. The frequently used function word and content word are incorporated
not only as the input features but also as the sentence structure constraint
for the GAN training. For the sentence generation with the related topics
decided by the user, the Named Entity Recognition (NER) information of the
input words is also used in the network training. We compared the proposed
method with the GAN-based sentence generation methods, and the experimental
results showed that the generated sentences using our method are more similar
to the original sentences of the same author based on the objective evaluation
such as BLEU and SimHash score.
Can Machine Learning Model with Static Features be Fooled: an Adversarial Machine Learning Approach
Authors: Rahim Taheri, Reza Javidan, Mohammad Shojafar, Vinod P, Mauro Conti
Date: 2019-04-20 00:00:00
Tasks: BIG-bench Machine Learning, Data Poisoning, Generative Adversarial Network, Malware Detection

The widespread adoption of smartphones dramatically increases the risk of attacks and the spread of mobile malware, especially on the Android platform. Machine learning-based solutions have been already used as a tool to supersede signature-based anti-malware systems. However, malware authors leverage features from malicious and legitimate samples to estimate statistical difference in-order to create adversarial examples. Hence, to evaluate the vulnerability of machine learning algorithms in malware detection, we propose five different attack scenarios to perturb malicious applications (apps). By doing this, the classification algorithm inappropriately fits the discriminant function on the set of data points, eventually yielding a higher misclassification rate. Further, to distinguish the adversarial examples from benign samples, we propose two defense mechanisms to counter attacks. To validate our attacks and solutions, we test our model on three different benchmark datasets. We also test our methods using various classifier algorithms and compare them with the state-of-the-art data poisoning method using the Jacobian matrix. Promising results show that generated adversarial samples can evade detection with a very high probability. Additionally, evasive variants generated by our attack models when used to harden the developed anti-malware system improves the detection rate up to 50% when using the Generative Adversarial Network (GAN) method.
Deep Learning for Large-Scale Real-World ACARS and ADS-B Radio Signal Classification
Authors: Shichuan Chen, Shilian Zheng, Lifeng Yang, Xiaoniu Yang
Date: 2019-04-20 00:00:00
Tasks: Classification, Deep Learning, General Classification, Management, Transfer Learning

Radio signal classification has a very wide range of applications in the field of wireless communications and electromagnetic spectrum management. In recent years, deep learning has been used to solve the problem of radio signal classification and has achieved good results. However, the radio signal data currently used is very limited in scale. In order to verify the performance of the deep learning-based radio signal classification on real-world radio signal data, in this paper we conduct experiments on large-scale real-world ACARS and ADS-B signal data with sample sizes of 900,000 and 13,000,000, respectively, and with categories of 3,143 and 5,157 respectively. We use the same Inception-Residual neural network model structure for ACARS signal classification and ADS-B signal classification to verify the ability of a single basic deep neural network model structure to process different types of radio signals, i.e., communication bursts in ACARS and pulse bursts in ADS-B. We build an experimental system for radio signal deep learning experiments. Experimental results show that the signal classification accuracy of ACARS and ADS-B is 98.1% and 96.3%, respectively. When the signal-to-noise ratio (with injected additive white Gaussian noise) is greater than 9 dB, the classification accuracy is greater than 92%. These experimental results validate the ability of deep learning to classify large-scale real-world radio signals. The results of the transfer learning experiment show that the model trained on large-scale ADS-B datasets is more conducive to the learning and training of new tasks than the model trained on small-scale datasets.
Specification-Driven Predictive Business Process Monitoring
Authors: Ario Santoso, Michael Felderer
Date: 2019-04-20 00:00:00
Tasks: Prediction

Predictive analysis in business process monitoring aims at forecasting the
future information of a running business process. The prediction is typically
made based on the model extracted from historical process execution logs (event
logs). In practice, different business domains might require different kinds of
predictions. Hence, it is important to have a means for properly specifying the
desired prediction tasks, and a mechanism to deal with these various prediction
tasks. Although there have been many studies in this area, they mostly focus on
a specific prediction task. This work introduces a language for specifying the
desired prediction tasks, and this language allows us to express various kinds
of prediction tasks. This work also presents a mechanism for automatically
creating the corresponding prediction model based on the given specification.
Differently from previous studies, instead of focusing on a particular
prediction task, we present an approach to deal with various prediction tasks
based on the given specification of the desired prediction tasks. We also
provide an implementation of the approach which is used to conduct experiments
using real-life event logs.
Multi-modal gated recurrent units for image description
Authors: Xuelong. Li, Aihong Yuan, Xiaoqiang Lu
Date: 2019-04-20 00:00:00
Tasks: Image Description, Sentence

Using a natural language sentence to describe the content of an image is a
challenging but very important task. It is challenging because a description
must not only capture objects contained in the image and the relationships
among them, but also be relevant and grammatically correct. In this paper a
multi-modal embedding model based on gated recurrent units (GRU) which can
generate variable-length description for a given image. In the training step,
we apply the convolutional neural network (CNN) to extract the image feature.
Then the feature is imported into the multi-modal GRU as well as the
corresponding sentence representations. The multi-modal GRU learns the
inter-modal relations between image and sentence. And in the testing step, when
an image is imported to our multi-modal GRU model, a sentence which describes
the image content is generated. The experimental results demonstrate that our
multi-modal GRU model obtains the state-of-the-art performance on Flickr8K,
Flickr30K and MS COCO datasets.
Distributed generation of privacy preserving data with user customization
Authors: Xiao Chen, Thomas Navidi, Stefano Ermon, Ram Rajagopal
Date: 2019-04-20 00:00:00
Tasks: Privacy Preserving

Distributed devices such as mobile phones can produce and store large amounts
of data that can enhance machine learning models; however, this data may
contain private information specific to the data owner that prevents the
release of the data. We wish to reduce the correlation between user-specific
private information and data while maintaining the useful information. Rather
than learning a large model to achieve privatization from end to end, we
introduce a decoupling of the creation of a latent representation and the
privatization of data that allows user-specific privatization to occur in a
distributed setting with limited computation and minimal disturbance on the
utility of the data. We leverage a Variational Autoencoder (VAE) to create a
compact latent representation of the data; however, the VAE remains fixed for
all devices and all possible private labels. We then train a small generative
filter to perturb the latent representation based on individual preferences
regarding the private and utility information. The small filter is trained by
utilizing a GAN-type robust optimization that can take place on a distributed
device. We conduct experiments on three popular datasets: MNIST, UCI-Adult, and
CelebA, and give a thorough evaluation including visualizing the geometry of
the latent embeddings and estimating the empirical mutual information to show
the effectiveness of our approach.
Challenges and Prospects in Vision and Language Research
Authors: Kushal Kafle, Robik Shrestha, Christopher Kanan
Date: 2019-04-19 00:00:00
Tasks: Natural Language Understanding

Language grounded image understanding tasks have often been proposed as a method for evaluating progress in artificial intelligence. Ideally, these tasks should test a plethora of capabilities that integrate computer vision, reasoning, and natural language understanding. However, rather than behaving as visual Turing tests, recent studies have demonstrated state-of-the-art systems are achieving good performance through flaws in datasets and evaluation procedures. We review the current state of affairs and outline a path forward.
Identifying Points of Interest and Similar Individuals from Raw GPS Data
Authors: Thiago Andrade, João Gama
Date: 2019-04-19 00:00:00

Smartphones and portable devices have become ubiquitous and part of
everyone's life. Due to the fact of its portability, these devices are perfect
to record individuals' traces and life-logging generating vast amounts of data
at low costs. These data is emerging as a new source for studies in human
mobility patterns raising the number of research projects and techniques aiming
to analyze and retrieve useful information from it. The aim of this paper is to
explore GPS raw data from different individuals in a community and apply data
mining algorithms to identify meaningful places in a region and describe user's
profiles and its similarities. We evaluate the proposed method with a
real-world dataset. The experimental results show that the steps performed to
identify points of interest (POIs) and further the similarity between the users
are quite satisfactory serving as a supplement for urban planning and social
networks.
Derivative-Free Global Optimization Algorithms: Bayesian Method and Lipschitzian Approaches
Authors: Jiawei Zhang
Date: 2019-04-19 00:00:00
Tasks: Deep Learning, global-optimization

In this paper, we will provide an introduction to the derivative-free
optimization algorithms which can be potentially applied to train deep learning
models. Existing deep learning model training is mostly based on the back
propagation algorithm, which updates the model variables layers by layers with
the gradient descent algorithm or its variants. However, the objective
functions of deep learning models to be optimized are usually non-convex and
the gradient descent algorithms based on the first-order derivative can get
stuck into the local optima very easily. To resolve such a problem, various
local or global optimization algorithms have been proposed, which can help
improve the training of deep learning models greatly. The representative
examples include the Bayesian methods, Shubert-Piyavskii algorithm, Direct,
LIPO, MCS, GA, SCE, DE, PSO, ES, CMA-ES, hill climbing and simulated annealing,
etc. One part of these algorithms will be introduced in this paper (including
the Bayesian method and Lipschitzian approaches, e.g., Shubert-Piyavskii
algorithm, Direct, LIPO and MCS), and the remaining algorithms (including the
population based optimization algorithms, e.g., GA, SCE, DE, PSO, ES and
CMA-ES, and random search algorithms, e.g., hill climbing and simulated
annealing) will be introduced in the follow-up paper [18] in detail.
Derivative-Free Global Optimization Algorithms: Population based Methods and Random Search Approaches
Authors: Jiawei Zhang
Date: 2019-04-19 00:00:00
Tasks: Deep Learning, global-optimization

In this paper, we will provide an introduction to the derivative-free
optimization algorithms which can be potentially applied to train deep learning
models. Existing deep learning model training is mostly based on the back
propagation algorithm, which updates the model variables layers by layers with
the gradient descent algorithm or its variants. However, the objective
functions of deep learning models to be optimized are usually non-convex and
the gradient descent algorithms based on the first-order derivative can get
stuck into the local optima very easily. To resolve such a problem, various
local or global optimization algorithms have been proposed, which can help
improve the training of deep learning models greatly. The representative
examples include the Bayesian methods, Shubert-Piyavskii algorithm, Direct,
LIPO, MCS, GA, SCE, DE, PSO, ES, CMA-ES, hill climbing and simulated annealing,
etc. This is a follow-up paper of [18], and we will introduce the population
based optimization algorithms, e.g., GA, SCE, DE, PSO, ES and CMA-ES, and
random search algorithms, e.g., hill climbing and simulated annealing, in this
paper. For the introduction to the other derivative-free optimization
algorithms, please refer to [18] for more information.
Minimax Optimal Online Stochastic Learning for Sequences of Convex Functions under Sub-Gradient Observation Failures
Authors: Hakan Gokcesu, Suleyman S. Kozat
Date: 2019-04-19 00:00:00

We study online convex optimization under stochastic sub-gradient observation
faults, where we introduce adaptive algorithms with minimax optimal regret
guarantees. We specifically study scenarios where our sub-gradient observations
can be noisy or even completely missing in a stochastic manner. To this end, we
propose algorithms based on sub-gradient descent method, which achieve tight
minimax optimal regret bounds. When necessary, these algorithms utilize
properties of the underlying stochastic settings to optimize their learning
rates (step sizes). These optimizations are the main factor in providing the
minimax optimal performance guarantees, especially when observations are
stochastically missing. However, in real world scenarios, these properties of
the underlying stochastic settings may not be revealed to the optimizer. For
such a scenario, we propose a blind algorithm that estimates these properties
empirically in a generally applicable manner. Through extensive experiments, we
show that this empirical approach is a natural combination of regular
stochastic gradient descent and the minimax optimal algorithms (which work best
for randomized and adversarial function sequences, respectively).