pwc-archive/papers-with-abstracts huggingface dataset:
Approximate message-passing for convex optimization with non-separable penalties
Authors: Andre Manoel, Florent Krzakala, Gaël Varoquaux, Bertrand Thirion, Lenka Zdeborová
Date: 2018-09-17 00:00:00

We introduce an iterative optimization scheme for convex objectives
consisting of a linear loss and a non-separable penalty, based on the
expectation-consistent approximation and the vector approximate message-passing
(VAMP) algorithm. Specifically, the penalties we approach are convex on a
linear transformation of the variable to be determined, a notable example being
total variation (TV). We describe the connection between message-passing
algorithms -- typically used for approximate inference -- and proximal methods
for optimization, and show that our scheme is, as VAMP, similar in nature to
the Peaceman-Rachford splitting, with the important difference that stepsizes
are set adaptively. Finally, we benchmark the performance of our VAMP-like
iteration in problems where TV penalties are useful, namely classification in
task fMRI and reconstruction in tomography, and show faster convergence than
that of state-of-the-art approaches such as FISTA and ADMM in most settings.
Adversarial Text Generation via Feature-Mover's Distance
Authors: Liqun Chen, Shuyang Dai, Chenyang Tao, Dinghan Shen, Zhe Gan, Haichao Zhang, Yizhe Zhang, Lawrence Carin
Date: 2018-09-17 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Adversarial Text, Style Transfer, Text Generation

Generative adversarial networks (GANs) have achieved significant success in generating real-valued data. However, the discrete nature of text hinders the application of GAN to text-generation tasks. Instead of using the standard GAN objective, we propose to improve text-generation GAN via a novel approach inspired by optimal transport. Specifically, we consider matching the latent feature distributions of real and synthetic sentences using a novel metric, termed the feature-mover's distance (FMD). This formulation leads to a highly discriminative critic and easy-to-optimize objective, overcoming the mode-collapsing and brittle-training problems in existing methods. Extensive experiments are conducted on a variety of tasks to evaluate the proposed model empirically, including unconditional text generation, style transfer from non-parallel text, and unsupervised cipher cracking. The proposed model yields superior performance, demonstrating wide applicability and effectiveness.
Optimal Matrix Momentum Stochastic Approximation and Applications to Q-learning
Authors: Adithya M. Devraj, Ana Bušić, Sean Meyn
Date: 2018-09-17 00:00:00
Tasks: Q-Learning, Reinforcement Learning, Stochastic Optimization

Acceleration is an increasingly common theme in the stochastic optimization
literature. The two most common examples are Nesterov's method, and Polyak's
momentum technique. In this paper two new algorithms are introduced for root
finding problems: 1) PolSA is a root finding algorithm with specially designed
matrix momentum, and 2) NeSA can be regarded as a variant of Nesterov's
algorithm, or a simplification of PolSA. The PolSA algorithm is new even in the
context of optimization (when cast as a root finding problem).
  The research surveyed in this paper is motivated by applications to
reinforcement learning. It is well known that most variants of TD- and
Q-learning may be cast as SA (stochastic approximation) algorithms, and the
tools from general SA theory can be used to investigate convergence and bounds
on convergence rate. In particular, the asymptotic variance is a common metric
of performance for SA algorithms, and is also one among many metrics used in
assessing the performance of stochastic optimization algorithms. There are two
well known SA techniques that are known to have optimal asymptotic variance:
the Ruppert-Polyak averaging technique, and stochastic Newton-Raphson (SNR).
The former algorithm can have extremely bad transient performance, and the
latter can be computationally expensive. It is demonstrated here that parameter
estimates from the new PolSA algorithm couple with those of the ideal (but more
complex) SNR algorithm. The new algorithm is thus a third approach to obtain
optimal asymptotic covariance.
  These strong results require assumptions on the model. A linearized model is
considered, and the noise is assumed to be a martingale difference sequence.
Numerical results are obtained in a non-linear setting that is the motivation
for this work: In PolSA implementations of Q-learning it is observed that
coupling occurs with SNR in this non-ideal setting.
Retrospective correction of Rigid and Non-Rigid MR motion artifacts using GANs
Authors: Karim Armanious, Sergios Gatidis, Konstantin Nikolaou, Bin Yang, Thomas Küstner
Date: 2018-09-17 00:00:00
Tasks: Diagnostic

Motion artifacts are a primary source of magnetic resonance (MR) image
quality deterioration with strong repercussions on diagnostic performance.
Currently, MR motion correction is carried out either prospectively, with the
help of motion tracking systems, or retrospectively by mainly utilizing
computationally expensive iterative algorithms. In this paper, we utilize a new
adversarial framework, titled MedGAN, for the joint retrospective correction of
rigid and non-rigid motion artifacts in different body regions and without the
need for a reference image. MedGAN utilizes a unique combination of
non-adversarial losses and a new generator architecture to capture the textures
and fine-detailed structures of the desired artifact-free MR images.
Quantitative and qualitative comparisons with other adversarial techniques have
illustrated the proposed model performance.
Learning Effective RGB-D Representations for Scene Recognition
Authors: Xinhang Song, Shuqiang Jiang, Luis Herranz, Chengpeng Chen
Date: 2018-09-17 00:00:00
Tasks: Scene Recognition, Video Recognition

Deep convolutional networks (CNN) can achieve impressive results on RGB scene
recognition thanks to large datasets such as Places. In contrast, RGB-D scene
recognition is still underdeveloped in comparison, due to two limitations of
RGB-D data we address in this paper. The first limitation is the lack of depth
data for training deep learning models. Rather than fine tuning or transferring
RGB-specific features, we address this limitation by proposing an architecture
and a two-step training approach that directly learns effective depth-specific
features using weak supervision via patches. The resulting RGB-D model also
benefits from more complementary multimodal features. Another limitation is the
short range of depth sensors (typically 0.5m to 5.5m), resulting in depth
images not capturing distant objects in the scenes that RGB images can. We show
that this limitation can be addressed by using RGB-D videos, where more
comprehensive depth information is accumulated as the camera travels across the
scene. Focusing on this scenario, we introduce the ISIA RGB-D video dataset to
evaluate RGB-D scene recognition with videos. Our video recognition
architecture combines convolutional and recurrent neural networks (RNNs) that
are trained in three steps with increasingly complex data to learn effective
features (i.e. patches, frames and sequences). Our approach obtains
state-of-the-art performances on RGB-D image (NYUD2 and SUN RGB-D) and video
(ISIA RGB-D) scene recognition.
Industrial Smoke Detection and Visualization
Authors: Yen-Chia Hsu, Paul Dille, Randy Sargent, Illah Nourbakhsh
Date: 2018-09-17 00:00:00
Tasks: Change Detection

As sensing technology proliferates and becomes affordable to the general
public, there is a growing trend in citizen science where scientists and
volunteers form a strong partnership in conducting scientific research
including problem finding, data collection, analysis, visualization, and
storytelling. Providing easy-to-use computational tools to support citizen
science has become an important issue. To raise the public awareness of
environmental science and improve the air quality in local areas, we are
currently collaborating with a local community in monitoring and documenting
fugitive emissions from a coke refinery. We have helped the community members
build a live camera system which captures and visualizes high resolution
timelapse imagery starting from November 2014. However, searching and
documenting smoke emissions manually from all video frames requires manpower
and takes an impractical investment of time. This paper describes a software
tool which integrates four features: (1) an algorithm based on change detection
and texture segmentation for identifying smoke emissions; (2) an interactive
timeline visualization providing indicators for seeking to interesting events;
(3) an autonomous fast-forwarding mode for skipping uninteresting timelapse
frames; and (4) a collection of animated smoke images generated automatically
according to the algorithm for documentation, presentation, storytelling, and
sharing. With the help of this tool, citizen scientists can now focus on the
content of the story instead of time-consuming and laborious works.
Learning to Collaborate: Multi-Scenario Ranking via Multi-Agent Reinforcement Learning
Authors: Jun Feng, Heng Li, Minlie Huang, Shichen Liu, Wenwu Ou, Zhirong Wang, Xiaoyan Zhu
Date: 2018-09-17 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Ranking is a fundamental and widely studied problem in scenarios such as
search, advertising, and recommendation. However, joint optimization for
multi-scenario ranking, which aims to improve the overall performance of
several ranking strategies in different scenarios, is rather untouched.
Separately optimizing each individual strategy has two limitations. The first
one is lack of collaboration between scenarios meaning that each strategy
maximizes its own objective but ignores the goals of other strategies, leading
to a sub-optimal overall performance. The second limitation is the inability of
modeling the correlation between scenarios meaning that independent
optimization in one scenario only uses its own user data but ignores the
context in other scenarios.
  In this paper, we formulate multi-scenario ranking as a fully cooperative,
partially observable, multi-agent sequential decision problem. We propose a
novel model named Multi-Agent Recurrent Deterministic Policy Gradient (MA-RDPG)
which has a communication component for passing messages, several private
actors (agents) for making actions for ranking, and a centralized critic for
evaluating the overall performance of the co-working actors. Each scenario is
treated as an agent (actor). Agents collaborate with each other by sharing a
global action-value function (the critic) and passing messages that encodes
historical information across scenarios. The model is evaluated with online
settings on a large E-commerce platform. Results show that the proposed model
exhibits significant improvements against baselines in terms of the overall
performance.
Sensor Transfer: Learning Optimal Sensor Effect Image Augmentation for Sim-to-Real Domain Adaptation
Authors: Alexandra Carlson, Katherine A. Skinner, Ram Vasudevan, Matthew Johnson-Roberson
Date: 2018-09-17 00:00:00
Tasks: Domain Adaptation, Image Augmentation, object-detection, Object Detection, Transfer Learning

Performance on benchmark datasets has drastically improved with advances in
deep learning. Still, cross-dataset generalization performance remains
relatively low due to the domain shift that can occur between two different
datasets. This domain shift is especially exaggerated between synthetic and
real datasets. Significant research has been done to reduce this gap,
specifically via modeling variation in the spatial layout of a scene, such as
occlusions, and scene environmental factors, such as time of day and weather
effects. However, few works have addressed modeling the variation in the sensor
domain as a means of reducing the synthetic to real domain gap. The camera or
sensor used to capture a dataset introduces artifacts into the image data that
are unique to the sensor model, suggesting that sensor effects may also
contribute to domain shift. To address this, we propose a learned augmentation
network composed of physically-based augmentation functions. Our proposed
augmentation pipeline transfers specific effects of the sensor model --
chromatic aberration, blur, exposure, noise, and color temperature -- from a
real dataset to a synthetic dataset. We provide experiments that demonstrate
that augmenting synthetic training datasets with the proposed learned
augmentation framework reduces the domain gap between synthetic and real
domains for object detection in urban driving scenes.
Unsupervised Sense-Aware Hypernymy Extraction
Authors: Dmitry Ustalov, Alexander Panchenko, Chris Biemann, Simone Paolo Ponzetto
Date: 2018-09-17 00:00:00

In this paper, we show how unsupervised sense representations can be used to
improve hypernymy extraction. We present a method for extracting disambiguated
hypernymy relationships that propagates hypernyms to sets of synonyms
(synsets), constructs embeddings for these sets, and establishes sense-aware
relationships between matching synsets. Evaluation on two gold standard
datasets for English and Russian shows that the method successfully recognizes
hypernymy relationships that cannot be found with standard Hearst patterns and
Wiktionary datasets for the respective languages.
Quantum Statistics-Inspired Neural Attention
Authors: Aristotelis Charalampous, Sotirios Chatzis
Date: 2018-09-17 00:00:00
Tasks: Machine Translation

Sequence-to-sequence (encoder-decoder) models with attention constitute a
cornerstone of deep learning research, as they have enabled unprecedented
sequential data modeling capabilities. This effectiveness largely stems from
the capacity of these models to infer salient temporal dynamics over long
horizons; these are encoded into the obtained neural attention (NA)
distributions. However, existing NA formulations essentially constitute
point-wise selection mechanisms over the observed source sequences; that is,
attention weights computation relies on the assumption that each source
sequence element is independent of the rest. Unfortunately, although
convenient, this assumption fails to account for higher-order dependencies
which might be prevalent in real-world data. This paper addresses these
limitations by leveraging Quantum-Statistical modeling arguments. Specifically,
our work broadens the notion of NA, by attempting to account for the case that
the NA model becomes inherently incapable of discerning between individual
source elements; this is assumed to be the case due to higher-order temporal
dynamics. On the contrary, we postulate that in some cases selection may be
feasible only at the level of pairs of source sequence elements. To this end,
we cast NA into inference of an attention density matrix (ADM) approximation.
We derive effective training and inference algorithms, and evaluate our
approach in the context of a machine translation (MT) application. We perform
experiments with challenging benchmark datasets. As we show, our approach
yields favorable outcomes in terms of several evaluation metrics.
From Same Photo: Cheating on Visual Kinship Challenges
Authors: Mitchell Dawson, Andrew Zisserman, Christoffer Nellåker
Date: 2018-09-17 00:00:00
Tasks: Kinship Verification

With the propensity for deep learning models to learn unintended signals from
data sets there is always the possibility that the network can `cheat' in order
to solve a task. In the instance of data sets for visual kinship verification,
one such unintended signal could be that the faces are cropped from the same
photograph, since faces from the same photograph are more likely to be from the
same family. In this paper we investigate the influence of this artefactual
data inference in published data sets for kinship verification.
  To this end, we obtain a large dataset, and train a CNN classifier to
determine if two faces are from the same photograph or not. Using this
classifier alone as a naive classifier of kinship, we demonstrate near state of
the art results on five public benchmark data sets for kinship verification -
achieving over 90% accuracy on one of them. Thus, we conclude that faces
derived from the same photograph are a strong inadvertent signal in all the
data sets we examined, and it is likely that the fraction of kinship explained
by existing kinship models is small.
Intermediate Deep Feature Compression: the Next Battlefield of Intelligent Sensing
Authors: Zhuo Chen, Weisi Lin, Shiqi Wang, Ling-Yu Duan, Alex C. Kot
Date: 2018-09-17 00:00:00
Tasks: Data Compression, Deep Learning, Feature Compression

The recent advances of hardware technology have made the intelligent analysis
equipped at the front-end with deep learning more prevailing and practical. To
better enable the intelligent sensing at the front-end, instead of compressing
and transmitting visual signals or the ultimately utilized top-layer deep
learning features, we propose to compactly represent and convey the
intermediate-layer deep learning features of high generalization capability, to
facilitate the collaborating approach between front and cloud ends. This
strategy enables a good balance among the computational load, transmission load
and the generalization ability for cloud servers when deploying the deep neural
networks for large scale cloud based visual analysis. Moreover, the presented
strategy also makes the standardization of deep feature coding more feasible
and promising, as a series of tasks can simultaneously benefit from the
transmitted intermediate layers. We also present the results for evaluation of
lossless deep feature compression with four benchmark data compression methods,
which provides meaningful investigations and baselines for future research and
standardization activities.
The Fast and the Flexible: training neural networks to learn to follow instructions from small data
Authors: Rezka Leonandya, Elia Bruni, Dieuwke Hupkes, Germán Kruszewski
Date: 2018-09-17 00:00:00
Proceeding: WS 2019 5
Tasks: Inductive Bias, Small Data Image Classification

Learning to follow human instructions is a long-pursued goal in artificial
intelligence. The task becomes particularly challenging if no prior knowledge
of the employed language is assumed while relying only on a handful of examples
to learn from. Work in the past has relied on hand-coded components or manually
engineered features to provide strong inductive biases that make learning in
such situations possible. In contrast, here we seek to establish whether this
knowledge can be acquired automatically by a neural network system through a
two phase training procedure: A (slow) offline learning stage where the network
learns about the general structure of the task and a (fast) online adaptation
phase where the network learns the language of a new given speaker. Controlled
experiments show that when the network is exposed to familiar instructions but
containing novel words, the model adapts very efficiently to the new
vocabulary. Moreover, even for human speakers whose language usage can depart
significantly from our artificial training language, our network can still make
use of its automatically acquired inductive bias to learn to follow
instructions more effectively.
Multi Modal Convolutional Neural Networks for Brain Tumor Segmentation
Authors: Mehmet Aygün, Yusuf Hüseyin Şahin, Gözde Ünal
Date: 2018-09-17 00:00:00
Tasks: Brain Tumor Segmentation, Segmentation, Tumor Segmentation, Video Recognition

In this work, we propose a multi-modal Convolutional Neural Network (CNN)
approach for brain tumor segmentation. We investigate how to combine different
modalities efficiently in the CNN framework.We adapt various fusion methods,
which are previously employed on video recognition problem, to the brain tumor
segmentation problem,and we investigate their efficiency in terms of memory and
performance.Our experiments, which are performed on BRATS dataset, lead us to
the conclusion that learning separate representations for each modality and
combining them for brain tumor segmentation could increase the performance of
CNN systems.
ADBSCAN: Adaptive Density-Based Spatial Clustering of Applications with Noise for Identifying Clusters with Varying Densities
Authors: Mohammad Mahmudur Rahman Khan, Md. Abu Bakr Siddique, Rezoana Bente Arif, Mahjabin Rahman Oishe
Date: 2018-09-17 00:00:00
Tasks: Clustering

Density-based spatial clustering of applications with noise (DBSCAN) is a
data clustering algorithm which has the high-performance rate for dataset where
clusters have the constant density of data points. One of the significant
attributes of this algorithm is noise cancellation. However, DBSCAN
demonstrates reduced performances for clusters with different densities.
Therefore, in this paper, an adaptive DBSCAN is proposed which can work
significantly well for identifying clusters with varying densities.
Study and Observation of the Variations of Accuracies for Handwritten Digits Recognition with Various Hidden Layers and Epochs using Neural Network Algorithm
Authors: Md. Abu Bakr Siddique, Mohammad Mahmudur Rahman Khan, Rezoana Bente Arif, Zahidun Ashrafi
Date: 2018-09-17 00:00:00
Tasks: Image Compression, Novelty Detection, speech-recognition, Speech Recognition, Stock Market Prediction

In recent days, Artificial Neural Network (ANN) can be applied to a vast
majority of fields including business, medicine, engineering, etc. The most
popular areas where ANN is employed nowadays are pattern and sequence
recognition, novelty detection, character recognition, regression analysis,
speech recognition, image compression, stock market prediction, Electronic
nose, security, loan applications, data processing, robotics, and control. The
benefits associated with its broad applications leads to increasing popularity
of ANN in the era of 21st Century. ANN confers many benefits such as organic
learning, nonlinear data processing, fault tolerance, and self-repairing
compared to other conventional approaches. The primary objective of this paper
is to analyze the influence of the hidden layers of a neural network over the
overall performance of the network. To demonstrate this influence, we applied
neural network with different layers on the MNIST dataset. Also, another goal
is to observe the variations of accuracies of ANN for different numbers of
hidden layers and epochs and to compare and contrast among them.
Study and Observation of the Variations of Accuracies for Handwritten Digits Recognition with Various Hidden Layers and Epochs using Convolutional Neural Network
Authors: Rezoana Bente Arif, Md. Abu Bakr Siddique, Mohammad Mahmudur Rahman Khan, Mahjabin Rahman Oishe
Date: 2018-09-17 00:00:00
Tasks: Face Recognition, image-classification, Image Classification, object-detection, Object Detection, Spam detection, speech-recognition, Speech Recognition

Nowadays, deep learning can be employed to a wide ranges of fields including
medicine, engineering, etc. In deep learning, Convolutional Neural Network
(CNN) is extensively used in the pattern and sequence recognition, video
analysis, natural language processing, spam detection, topic categorization,
regression analysis, speech recognition, image classification, object
detection, segmentation, face recognition, robotics, and control. The benefits
associated with its near human level accuracies in large applications lead to
the growing acceptance of CNN in recent years. The primary contribution of this
paper is to analyze the impact of the pattern of the hidden layers of a CNN
over the overall performance of the network. To demonstrate this influence, we
applied neural network with different layers on the Modified National Institute
of Standards and Technology (MNIST) dataset. Also, is to observe the variations
of accuracies of the network for various numbers of hidden layers and epochs
and to make comparison and contrast among them. The system is trained utilizing
stochastic gradient and backpropagation algorithm and tested with feedforward
algorithm.