pwc-archive/papers-with-abstracts huggingface dataset:
Semi-Supervised Multitask Learning on Multispectral Satellite Images Using Wasserstein Generative Adversarial Networks (GANs) for Predicting Poverty
Authors: Anthony Perez, Swetava Ganguli, Stefano Ermon, George Azzari, Marshall Burke, David Lobell
Date: 2019-02-13 00:00:00

Obtaining reliable data describing local poverty metrics at a granularity
that is informative to policy-makers requires expensive and logistically
difficult surveys, particularly in the developing world. Not surprisingly, the
poverty stricken regions are also the ones which have a high probability of
being a war zone, have poor infrastructure and sometimes have governments that
do not cooperate with internationally funded development efforts. We train a
CNN on free and publicly available daytime satellite images of the African
continent from Landsat 7 to build a model for predicting local economic
livelihoods. Only 5% of the satellite images can be associated with labels
(which are obtained from DHS Surveys) and thus a semi-supervised approach using
a GAN (similar to the approach of Salimans, et al. (2016)), albeit with a more
stable-to-train flavor of GANs called the Wasserstein GAN regularized with
gradient penalty(Gulrajani, et al. (2017)) is used. The method of multitask
learning is employed to regularize the network and also create an end-to-end
model for the prediction of multiple poverty metrics.
A Generative Map for Image-based Camera Localization
Authors: Mingpan Guo, Stefan Matthes, Jiaojiao Ye, Hao Shen
Date: 2019-02-18 00:00:00
Tasks: Camera Localization, regression, Visual Localization, Visual Odometry

In image-based camera localization systems, information about the environment
is usually stored in some representation, which can be referred to as a map.
Conventionally, most maps are built upon hand-crafted features. Recently,
neural networks have attracted attention as a data-driven map representation,
and have shown promising results in visual localization. However, these neural
network maps are generally hard to interpret by human. A readable map is not
only accessible to humans, but also provides a way to be verified when the
ground truth pose is unavailable. To tackle this problem, we propose Generative
Map, a new framework for learning human-readable neural network maps, by
combining a generative model with the Kalman filter, which also allows it to
incorporate additional sensor information such as stereo visual odometry. For
evaluation, we use real world images from the 7-Scenes and Oxford RobotCar
datasets. We demonstrate that our Generative Map can be queried with a pose of
interest from the test sequence to predict an image, which closely resembles
the true scene. For localization, we show that Generative Map achieves
comparable performance with current regression models. Moreover, our framework
is trained completely from scratch, unlike regression models which rely on
large ImageNet pretrained networks.
Cloud-based or On-device: An Empirical Study of Mobile Deep Inference
Authors: Tian Guo
Date: 2017-07-14 00:00:00
Tasks: Deep Learning, GPU

Modern mobile applications are benefiting significantly from the advancement
in deep learning, e.g., implementing real-time image recognition and
conversational system. Given a trained deep learning model, applications
usually need to perform a series of matrix operations based on the input data,
in order to infer possible output values. Because of computational complexity
and size constraints, these trained models are often hosted in the cloud. To
utilize these cloud-based models, mobile apps will have to send input data over
the network. While cloud-based deep learning can provide reasonable response
time for mobile apps, it restricts the use case scenarios, e.g. mobile apps
need to have network access. With mobile specific deep learning optimizations,
it is now possible to employ on-device inference. However, because mobile
hardware, such as GPU and memory size, can be very limited when compared to its
desktop counterpart, it is important to understand the feasibility of this new
on-device deep learning inference architecture. In this paper, we empirically
evaluate the inference performance of three Convolutional Neural Networks
(CNNs) using a benchmark Android application we developed. Our measurement and
analysis suggest that on-device inference can cost up to two orders of
magnitude greater response time and energy when compared to cloud-based
inference, and that loading model and computing probability are two performance
bottlenecks for on-device deep inferences.
Usage of analytic hierarchy process for steganographic inserts detection in images
Authors: S. V. Belim, D. E. Vilkhovskiy
Date: 2018-12-25 00:00:00

This article presents the method of steganography detection, which is formed
by replacing the least significant bit (LSB). Detection is performed by
dividing the image into layers and making an analysis of zero-layer of adjacent
bits for every bit. First-layer and second-layer are analyzed too. Hierarchies
analysis method is used for making decision if current bit is changed.
Weighting coefficients as part of the analytic hierarchy process are formed on
the values of bits. Then a matrix of corrupted pixels is generated.
Visualization of matrix with corrupted pixels allows to determine size,
location and presence of the embedded message. Computer experiment was
performed. Message was embedded in a bounded rectangular area of the image.
This method demonstrated efficiency even at low filling container, less than
10\%. Widespread statistical methods are unable to detect this steganographic
insert. The location and size of the embedded message can be determined with an
error which is not exceeding to five pixels.
A new Backdoor Attack in CNNs by training set corruption without label poisoning
Authors: Mauro Barni, Kassem Kallas, Benedetta Tondi
Date: 2019-02-12 00:00:00
Tasks: Backdoor Attack, General Classification

Backdoor attacks against CNNs represent a new threat against deep learning
systems, due to the possibility of corrupting the training set so to induce an
incorrect behaviour at test time. To avoid that the trainer recognises the
presence of the corrupted samples, the corruption of the training set must be
as stealthy as possible. Previous works have focused on the stealthiness of the
perturbation injected into the training samples, however they all assume that
the labels of the corrupted samples are also poisoned. This greatly reduces the
stealthiness of the attack, since samples whose content does not agree with the
label can be identified by visual inspection of the training set or by running
a pre-classification step. In this paper we present a new backdoor attack
without label poisoning Since the attack works by corrupting only samples of
the target class, it has the additional advantage that it does not need to
identify beforehand the class of the samples to be attacked at test time.
Results obtained on the MNIST digits recognition task and the traffic signs
classification task show that backdoor attacks without label poisoning are
indeed possible, thus raising a new alarm regarding the use of deep learning in
security-critical applications.
Global Stock Market Prediction Based on Stock Chart Images Using Deep Q-Network
Authors: Jinho Lee, Raehyun Kim, Yookyung Koh, Jaewoo Kang
Date: 2019-02-28 00:00:00
Tasks: Stock Market Prediction

We applied Deep Q-Network with a Convolutional Neural Network function
approximator, which takes stock chart images as input, for making global stock
market predictions. Our model not only yields profit in the stock market of the
country where it was trained but generally yields profit in global stock
markets. We trained our model only in the US market and tested it in 31
different countries over 12 years. The portfolios constructed based on our
model's output generally yield about 0.1 to 1.0 percent return per transaction
prior to transaction costs in 31 countries. The results show that there are
some patterns on stock chart image, that tend to predict the same future stock
price movements across global stock markets. Moreover, the results show that
future stock prices can be predicted even if the training and testing
procedures are done in different countries. Training procedure could be done in
relatively large and liquid markets (e.g., USA) and tested in small markets.
This result demonstrates that artificial intelligence based stock price
forecasting models can be used in relatively small markets (emerging countries)
even though they do not have a sufficient amount of data for training.
PixelSteganalysis: Destroying Hidden Information with a Low Degree of Visual Degradation
Authors: Dahuin Jung, Ho Bae, Hyun-Soo Choi, Sungroh Yoon
Date: 2019-01-30 00:00:00
Tasks: Deep Learning, Image Steganography, Steganalysis

Steganography is the science of unnoticeably concealing a secret message
within a certain image, called a cover image. The cover image with the secret
message is called a stego image. Steganography is commonly used for illegal
purposes such as terrorist activities and pornography. To thwart covert
communications and transactions, attacking algorithms against steganography,
called steganalysis, exist. Currently, there are many studies implementing deep
learning to the steganography algorithm. However, conventional steganalysis is
no longer effective for deep learning based steganography algorithms. Our
framework is the first one to disturb covert communications and transactions
via the recent deep learning-based steganography algorithms. We first extract a
sophisticated pixel distribution of the potential stego image from the
auto-regressive model induced by deep learning. Using the extracted pixel
distributions, we detect whether an image is the stego or not at the pixel
level. Each pixel value is adjusted as required and the adjustment induces an
effective removal of the secret image. Because the decoding method of deep
learning-based steganography algorithms is approximate (lossy), which is
different from the conventional steganography, we propose a new quantitative
metric that is more suitable for measuring the accurate effect. We evaluate our
method using three public benchmarks in comparison with a conventional
steganalysis method and show up to a 20% improvement in terms of decoding rate.
ASTER: An Attentional Scene Text Recognizer with Flexible Rectification
Authors: Baoguang Shi, Mingkun Yang, Xinggang Wang, Pengyuan Lyu, Cong Yao, and Xiang Bai
Date: 2018-06-25 00:00:00
Proceeding: good 2018 6
Tasks: Optical Character Recognition, Optical Character Recognition (OCR), Scene Text Recognition

SCENE text recognition has attracted great interest from
the academia and the industry in recent years owing to
its importance in a wide range of applications. Despite the
maturity of Optical Character Recognition (OCR) systems
dedicated to document text, scene text recognition remains
a challenging problem. The large variations in background,
appearance, and layout pose significant challenges, which
the traditional OCR methods cannot handle effectively.
Recent advances in scene text recognition are driven
by the success of deep learning-based recognition models.
Among them are methods that recognize text by characters
using convolutional neural networks (CNN), methods that
classify words with CNNs [24], [26], and methods that
recognize character sequences using a combination of a
CNN and a recurrent neural network (RNN) [54]. In spite
of their success, these methods do not explicitly address the
problem of irregular text, which is text that is not horizontal
and frontal, has curved layout, etc. Instances of irregular
text frequently appear in natural scenes. As exemplified
in Figure 1, typical cases include oriented text, perspective
text [49], and curved text. Designed without the invariance
to such irregularities, previous methods often struggle in
recognizing such text instances.
A sticky HDP-HMM with application to speaker diarization
Authors: Emily B. Fox, Erik B. Sudderth, Michael. I. Jordan, Alan S. Willsky
Date: 2009-05-15 00:00:00
Tasks: speaker-diarization, Speaker Diarization

We consider the problem of speaker diarization, the problem of segmenting an
audio recording of a meeting into temporal segments corresponding to individual
speakers. The problem is rendered particularly difficult by the fact that we
are not allowed to assume knowledge of the number of people participating in
the meeting. To address this problem, we take a Bayesian nonparametric approach
to speaker diarization that builds on the hierarchical Dirichlet process hidden
Markov model (HDP-HMM) of Teh et al. [J. Amer. Statist. Assoc. 101 (2006)
1566--1581]. Although the basic HDP-HMM tends to over-segment the audio
data---creating redundant states and rapidly switching among them---we describe
an augmented HDP-HMM that provides effective control over the switching rate.
We also show that this augmentation makes it possible to treat emission
distributions nonparametrically. To scale the resulting architecture to
realistic diarization problems, we develop a sampling algorithm that employs a
truncated approximation of the Dirichlet process to jointly resample the full
state sequence, greatly improving mixing rates. Working with a benchmark NIST
data set, we show that our Bayesian nonparametric architecture yields
state-of-the-art speaker diarization results.
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Authors: Philipp Krähenbühl, Vladlen Koltun
Date: 2012-10-20 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Most state-of-the-art techniques for multi-class image segmentation and
labeling use conditional random fields defined over pixels or image regions.
While region-level models often feature dense pairwise connectivity,
pixel-level models are considerably larger and have only permitted sparse graph
structures. In this paper, we consider fully connected CRF models defined on
the complete set of pixels in an image. The resulting graphs have billions of
edges, making traditional inference algorithms impractical. Our main
contribution is a highly efficient approximate inference algorithm for fully
connected CRF models in which the pairwise edge potentials are defined by a
linear combination of Gaussian kernels. Our experiments demonstrate that dense
connectivity at the pixel level substantially improves segmentation and
labeling accuracy.
SafeDrive: Enhancing Lane Appearance for Autonomous and Assisted Driving Under Limited Visibility
Authors: Jiawei Mo, Junaed Sattar
Date: 2018-07-24 00:00:00
Tasks: Autonomous Driving, Lane Detection, Visual Tracking

Autonomous detection of lane markers improves road safety, and purely visual
tracking is desirable for widespread vehicle compatibility and reducing sensor
intrusion, cost, and energy consumption. However, visual approaches are often
ineffective because of a number of factors; e.g., occlusion, poor weather
conditions, and paint wear-off. We present an approach to enhance lane marker
appearance for assisted and autonomous driving, particularly under poor
visibility. Our method, named SafeDrive, attempts to improve visual lane
detection approaches in drastically degraded visual conditions. SafeDrive finds
lane markers in alternate imagery of the road at the vehicle's location and
reconstructs a sparse 3D model of the surroundings. By estimating the geometric
relationship between this 3D model and the current view, the lane markers are
projected onto the visual scene; any lane detection algorithm can be
subsequently used to detect lanes in the resulting image. SafeDrive does not
require additional sensors other than vision and location data. We demonstrate
the effectiveness of our approach on a number of test cases obtained from
actual driving data recorded in urban settings.
Multi-Object Representation Learning with Iterative Variational Inference
Authors: Klaus Greff, Raphaël Lopez Kaufman, Rishabh Kabra, Nick Watters, Chris Burgess, Daniel Zoran, Loic Matthey, Matthew Botvinick, Alexander Lerchner
Date: 2019-03-01 00:00:00
Tasks: Object, Representation Learning, Systematic Generalization, Unsupervised Object Segmentation, Variational Inference

Human perception is structured around objects which form the basis for our higher-level cognition and impressive systematic generalization abilities. Yet most work on representation learning focuses on feature learning without even considering multiple objects, or treats segmentation as an (often supervised) preprocessing step. Instead, we argue for the importance of learning to segment and represent objects jointly. We demonstrate that, starting from the simple assumption that a scene is composed of multiple entities, it is possible to learn to segment images into interpretable objects with disentangled representations. Our method learns -- without supervision -- to inpaint occluded parts, and extrapolates to scenes with more objects and to unseen objects with novel feature combinations. We also show that, due to the use of iterative variational inference, our system is able to learn multi-modal posteriors for ambiguous inputs and extends naturally to sequences.
Mask Scoring R-CNN
Authors: Zhaojin Huang, Lichao Huang, Yongchao Gong, Chang Huang, Xinggang Wang
Date: 2019-03-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: General Classification, Instance Segmentation, Segmentation, Semantic Segmentation

Letting a deep network be aware of the quality of its own predictions is an
interesting yet important problem. In the task of instance segmentation, the
confidence of instance classification is used as mask quality score in most
instance segmentation frameworks. However, the mask quality, quantified as the
IoU between the instance mask and its ground truth, is usually not well
correlated with classification score. In this paper, we study this problem and
propose Mask Scoring R-CNN which contains a network block to learn the quality
of the predicted instance masks. The proposed network block takes the instance
feature and the corresponding predicted mask together to regress the mask IoU.
The mask scoring strategy calibrates the misalignment between mask quality and
mask score, and improves instance segmentation performance by prioritizing more
accurate mask predictions during COCO AP evaluation. By extensive evaluations
on the COCO dataset, Mask Scoring R-CNN brings consistent and noticeable gain
with different models, and outperforms the state-of-the-art Mask R-CNN. We hope
our simple and effective approach will provide a new direction for improving
instance segmentation. The source code of our method is available at
\url{https://github.com/zjhuang22/maskscoring_rcnn}.
Octree guided CNN with Spherical Kernels for 3D Point Clouds
Authors: Huan Lei, Naveed Akhtar, Ajmal Mian
Date: 2019-02-28 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Object Classification, 3D Part Segmentation, Translation

We propose an octree guided neural network architecture and spherical
convolutional kernel for machine learning from arbitrary 3D point clouds. The
network architecture capitalizes on the sparse nature of irregular point
clouds, and hierarchically coarsens the data representation with space
partitioning. At the same time, the proposed spherical kernels systematically
quantize point neighborhoods to identify local geometric structures in the
data, while maintaining the properties of translation-invariance and asymmetry.
We specify spherical kernels with the help of network neurons that in turn are
associated with spatial locations. We exploit this association to avert dynamic
kernel generation during network training that enables efficient learning with
high resolution point clouds. The effectiveness of the proposed technique is
established on the benchmark tasks of 3D object classification and
segmentation, achieving new state-of-the-art on ShapeNet and RueMonge2014
datasets.
A Behavioral Approach to Visual Navigation with Graph Localization Networks
Authors: Kevin Chen, Juan Pablo de Vicente, Gabriel Sepulveda, Fei Xia, Alvaro Soto, Marynel Vazquez, Silvio Savarese
Date: 2019-03-01 00:00:00
Tasks: Navigate, Visual Navigation

Inspired by research in psychology, we introduce a behavioral approach for
visual navigation using topological maps. Our goal is to enable a robot to
navigate from one location to another, relying only on its visual input and the
topological map of the environment. We propose using graph neural networks for
localizing the agent in the map, and decompose the action space into primitive
behaviors implemented as convolutional or recurrent neural networks. Using the
Gibson simulator, we verify that our approach outperforms relevant baselines
and is able to navigate in both seen and unseen environments.
Deep Learning for Multiple-Image Super-Resolution
Authors: Michal Kawulok, Pawel Benecki, Szymon Piechaczek, Krzysztof Hrynczenko, Daniel Kostrzewa, Jakub Nalepa
Date: 2019-03-01 00:00:00
Tasks: Deep Learning, Image Super-Resolution, Relation, Super-Resolution

Super-resolution reconstruction (SRR) is a process aimed at enhancing spatial
resolution of images, either from a single observation, based on the learned
relation between low and high resolution, or from multiple images presenting
the same scene. SRR is particularly valuable, if it is infeasible to acquire
images at desired resolution, but many images of the same scene are available
at lower resolution---this is inherent to a variety of remote sensing
scenarios. Recently, we have witnessed substantial improvement in single-image
SRR attributed to the use of deep neural networks for learning the relation
between low and high resolution. Importantly, deep learning has not been
exploited for multiple-image SRR, which benefits from information fusion and in
general allows for achieving higher reconstruction accuracy. In this letter, we
introduce a new method which combines the advantages of multiple-image fusion
with learning the low-to-high resolution mapping using deep networks. The
reported experimental results indicate that our algorithm outperforms the
state-of-the-art SRR methods, including these that operate from a single image,
as well as those that perform multiple-image fusion.
Using natural language processing techniques to extract information on the properties and functionalities of energetic materials from large text corpora
Authors: Daniel C. Elton, Dhruv Turakhia, Nischal Reddy, Zois Boukouvalas, Mark D. Fuge, Ruth M. Doherty, Peter W. Chung
Date: 2019-03-01 00:00:00
Tasks: Articles, Word Embeddings

The number of scientific journal articles and reports being published about
energetic materials every year is growing exponentially, and therefore
extracting relevant information and actionable insights from the latest
research is becoming a considerable challenge. In this work we explore how
techniques from natural language processing and machine learning can be used to
automatically extract chemical insights from large collections of documents. We
first describe how to download and process documents from a variety of sources
- journal articles, conference proceedings (including NTREM), the US Patent &
Trademark Office, and the Defense Technical Information Center archive on
archive.org. We present a custom NLP pipeline which uses open source NLP tools
to identify the names of chemical compounds and relates them to function words
("underwater", "rocket", "pyrotechnic") and property words ("elastomer",
"non-toxic"). After explaining how word embeddings work we compare the utility
of two popular word embeddings - word2vec and GloVe. Chemical-chemical and
chemical-application relationships are obtained by doing computations with word
vectors. We show that word embeddings capture latent information about
energetic materials, so that related materials appear close together in the
word embedding space.