pwc-archive/papers-with-abstracts huggingface dataset:
Structural Consistency and Controllability for Diverse Colorization
Authors: Safa Messaoud, David Forsyth, Alexander G. Schwing
Date: 2018-09-06 00:00:00
Proceeding: ECCV 2018 9
Tasks: Colorization, Diversity

Colorizing a given gray-level image is an important task in the media and
advertising industry. Due to the ambiguity inherent to colorization (many
shades are often plausible), recent approaches started to explicitly model
diversity. However, one of the most obvious artifacts, structural
inconsistency, is rarely considered by existing methods which predict
chrominance independently for every pixel. To address this issue, we develop a
conditional random field based variational auto-encoder formulation which is
able to achieve diversity while taking into account structural consistency.
Moreover, we introduce a controllability mecha- nism that can incorporate
external constraints from diverse sources in- cluding a user interface.
Compared to existing baselines, we demonstrate that our method obtains more
diverse and globally consistent coloriza- tions on the LFW, LSUN-Church and
ILSVRC-2015 datasets.
Labeling Panoramas with Spherical Hourglass Networks
Authors: Carlos Esteves, Kostas Daniilidis, Ameesh Makadia
Date: 2018-09-06 00:00:00
Tasks: Semantic Segmentation

With the recent proliferation of consumer-grade 360{\deg} cameras, it is
worth revisiting visual perception challenges with spherical cameras given the
potential benefit of their global field of view. To this end we introduce a
spherical convolutional hourglass network (SCHN) for the dense labeling on the
sphere. The SCHN is invariant to camera orientation (lifting the usual
requirement for `upright' panoramic images), and its design is scalable for
larger practical datasets. Initial experiments show promising results on a
spherical semantic segmentation task.
Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning
Authors: Tom Zahavy, Matan Haroush, Nadav Merlis, Daniel J. Mankowitz, Shie Mannor
Date: 2018-09-06 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), text-based games

Learning how to act when there are many available actions in each state is a
challenging task for Reinforcement Learning (RL) agents, especially when many
of the actions are redundant or irrelevant. In such cases, it is sometimes
easier to learn which actions not to take. In this work, we propose the
Action-Elimination Deep Q-Network (AE-DQN) architecture that combines a Deep RL
algorithm with an Action Elimination Network (AEN) that eliminates sub-optimal
actions. The AEN is trained to predict invalid actions, supervised by an
external elimination signal provided by the environment. Simulations
demonstrate a considerable speedup and added robustness over vanilla DQN in
text-based games with over a thousand discrete actions.
ANS: Adaptive Network Scaling for Deep Rectifier Reinforcement Learning Models
Authors: Yueh-Hua Wu, Fan-Yun Sun, Yen-Yu Chang, Shou-De Lin
Date: 2018-09-06 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This work provides a thorough study on how reward scaling can affect
performance of deep reinforcement learning agents. In particular, we would like
to answer the question that how does reward scaling affect non-saturating ReLU
networks in RL? This question matters because ReLU is one of the most effective
activation functions for deep learning models. We also propose an Adaptive
Network Scaling framework to find a suitable scale of the rewards during
learning for better performance. We conducted empirical studies to justify the
solution.
Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network
Authors: Daan de Geus, Panagiotis Meletis, Gijs Dubbelman
Date: 2018-09-06 00:00:00
Proceeding: CoRR 2019 2
Tasks: Instance Segmentation, Panoptic Segmentation, Segmentation, Semantic Segmentation

We present a single network method for panoptic segmentation. This method
combines the predictions from a jointly trained semantic and instance
segmentation network using heuristics. Joint training is the first step towards
an end-to-end panoptic segmentation network and is faster and more memory
efficient than training and predicting with two networks, as done in previous
work. The architecture consists of a ResNet-50 feature extractor shared by the
semantic segmentation and instance segmentation branch. For instance
segmentation, a Mask R-CNN type of architecture is used, while the semantic
segmentation branch is augmented with a Pyramid Pooling Module. Results for
this method are submitted to the COCO and Mapillary Joint Recognition Challenge
2018. Our approach achieves a PQ score of 17.6 on the Mapillary Vistas
validation set and 27.2 on the COCO test-dev set.
Deep Audio-Visual Speech Recognition
Authors: Triantafyllos Afouras, Joon Son Chung, Andrew Senior, Oriol Vinyals, Andrew Zisserman
Date: 2018-09-06 00:00:00
Tasks: Audio-Visual Speech Recognition, Automatic Speech Recognition (ASR), Lipreading, Lip Reading, speech-recognition, Speech Recognition, Visual Speech Recognition

The goal of this work is to recognise phrases and sentences being spoken by a
talking face, with or without the audio. Unlike previous works that have
focussed on recognising a limited number of words or phrases, we tackle lip
reading as an open-world problem - unconstrained natural language sentences,
and in the wild videos. Our key contributions are: (1) we compare two models
for lip reading, one using a CTC loss, and the other using a
sequence-to-sequence loss. Both models are built on top of the transformer
self-attention architecture; (2) we investigate to what extent lip reading is
complementary to audio speech recognition, especially when the audio signal is
noisy; (3) we introduce and publicly release a new dataset for audio-visual
speech recognition, LRS2-BBC, consisting of thousands of natural sentences from
British television. The models that we train surpass the performance of all
previous work on a lip reading benchmark dataset by a significant margin.
A Memory-Network Based Solution for Multivariate Time-Series Forecasting
Authors: Yen-Yu Chang, Fan-Yun Sun, Yueh-Hua Wu, Shou-De Lin
Date: 2018-09-06 00:00:00
Tasks: Multivariate Time Series Forecasting, Question Answering, Time Series, Time Series Analysis, Time Series Forecasting

Multivariate time series forecasting is extensively studied throughout the
years with ubiquitous applications in areas such as finance, traffic,
environment, etc. Still, concerns have been raised on traditional methods for
incapable of modeling complex patterns or dependencies lying in real word data.
To address such concerns, various deep learning models, mainly Recurrent Neural
Network (RNN) based methods, are proposed. Nevertheless, capturing extremely
long-term patterns while effectively incorporating information from other
variables remains a challenge for time-series forecasting. Furthermore,
lack-of-explainability remains one serious drawback for deep neural network
models. Inspired by Memory Network proposed for solving the question-answering
task, we propose a deep learning based model named Memory Time-series network
(MTNet) for time series forecasting. MTNet consists of a large memory
component, three separate encoders, and an autoregressive component to train
jointly. Additionally, the attention mechanism designed enable MTNet to be
highly interpretable. We can easily tell which part of the historic data is
referenced the most.
Are adversarial examples inevitable?
Authors: Ali Shafahi, W. Ronny Huang, Christoph Studer, Soheil Feizi, Tom Goldstein
Date: 2018-09-06 00:00:00
Proceeding: ICLR 2019 5

A wide range of defenses have been proposed to harden neural networks against adversarial attacks. However, a pattern has emerged in which the majority of adversarial defenses are quickly broken by new attacks. Given the lack of success at generating robust defenses, we are led to ask a fundamental question: Are adversarial attacks inevitable? This paper analyzes adversarial examples from a theoretical perspective, and identifies fundamental bounds on the susceptibility of a classifier to adversarial attacks. We show that, for certain classes of problems, adversarial examples are inescapable. Using experiments, we explore the implications of theoretical guarantees for real-world problems and discuss how factors such as dimensionality and image complexity limit a classifier's robustness against adversarial examples.
Uncovering divergent linguistic information in word embeddings with lessons for intrinsic and extrinsic evaluation
Authors: Mikel Artetxe, Gorka Labaka, Iñigo Lopez-Gazpio, Eneko Agirre
Date: 2018-09-06 00:00:00
Proceeding: CONLL 2018 10
Tasks: Word Embeddings

Following the recent success of word embeddings, it has been argued that
there is no such thing as an ideal representation for words, as different
models tend to capture divergent and often mutually incompatible aspects like
semantics/syntax and similarity/relatedness. In this paper, we show that each
embedding model captures more information than directly apparent. A linear
transformation that adjusts the similarity order of the model without any
external resource can tailor it to achieve better results in those aspects,
providing a new perspective on how embeddings encode divergent linguistic
information. In addition, we explore the relation between intrinsic and
extrinsic evaluation, as the effect of our transformations in downstream tasks
is higher for unsupervised systems than for supervised ones.
Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models
Authors: Tong Niu, Mohit Bansal
Date: 2018-09-06 00:00:00
Proceeding: CONLL 2018 10
Tasks: Sensitivity

We present two categories of model-agnostic adversarial strategies that
reveal the weaknesses of several generative, task-oriented dialogue models:
Should-Not-Change strategies that evaluate over-sensitivity to small and
semantics-preserving edits, as well as Should-Change strategies that test if a
model is over-stable against subtle yet semantics-changing modifications. We
next perform adversarial training with each strategy, employing a max-margin
approach for negative generative examples. This not only makes the target
dialogue model more robust to the adversarial inputs, but also helps it perform
significantly better on the original inputs. Moreover, training on all
strategies combined achieves further improvements, achieving a new
state-of-the-art performance on the original task (also verified via human
evaluation). In addition to adversarial training, we also address the
robustness task at the model-level, by feeding it subword units as both inputs
and outputs, and show that the resulting model is equally competitive, requires
only 1/4 of the original vocabulary size, and is robust to one of the
adversarial strategies (to which the original model is vulnerable) even without
adversarial training.
IDSGAN: Generative Adversarial Networks for Attack Generation against Intrusion Detection
Authors: Zilong Lin, Yong Shi, Zhi Xue
Date: 2018-09-06 00:00:00
Tasks: Adversarial Attack, Intrusion Detection

As an essential tool in security, the intrusion detection system bears the responsibility of the defense to network attacks performed by malicious traffic. Nowadays, with the help of machine learning algorithms, intrusion detection systems develop rapidly. However, the robustness of this system is questionable when it faces adversarial attacks. For the robustness of detection systems, more potential attack approaches are under research. In this paper, a framework of the generative adversarial networks, called IDSGAN, is proposed to generate the adversarial malicious traffic records aiming to attack intrusion detection systems by deceiving and evading the detection. Given that the internal structure and parameters of the detection system are unknown to attackers, the adversarial attack examples perform the black-box attacks against the detection system. IDSGAN leverages a generator to transform original malicious traffic records into adversarial malicious ones. A discriminator classifies traffic examples and dynamically learns the real-time black-box detection system. More significantly, the restricted modification mechanism is designed for the adversarial generation to preserve original attack functionalities of adversarial traffic records. The effectiveness of the model is indicated by attacking multiple algorithm-based detection models with different attack categories. The robustness is verified by changing the number of the modified features. A comparative experiment with adversarial attack baselines demonstrates the superiority of our model.
Multiple Object Tracking in Urban Traffic Scenes with a Multiclass Object Detector
Authors: Hui-Lee Ooi, Guillaume-Alexandre Bilodeau, Nicolas Saunier, David-Alexandre Beaupré
Date: 2018-09-06 00:00:00
Tasks: General Classification, Multiple Object Tracking, Object, Object Tracking, Position

Multiple object tracking (MOT) in urban traffic aims to produce the
trajectories of the different road users that move across the field of view
with different directions and speeds and that can have varying appearances and
sizes. Occlusions and interactions among the different objects are expected and
common due to the nature of urban road traffic. In this work, a tracking
framework employing classification label information from a deep learning
detection approach is used for associating the different objects, in addition
to object position and appearances. We want to investigate the performance of a
modern multiclass object detector for the MOT task in traffic scenes. Results
show that the object labels improve tracking performance, but that the output
of object detectors are not always reliable.
ARCHER: Aggressive Rewards to Counter bias in Hindsight Experience Replay
Authors: Sameera Lanka, Tianfu Wu
Date: 2018-09-06 00:00:00
Tasks: continuous-control, Continuous Control, Deep Reinforcement Learning, Reinforcement Learning, Reinforcement Learning (RL)

Experience replay is an important technique for addressing
sample-inefficiency in deep reinforcement learning (RL), but faces difficulty
in learning from binary and sparse rewards due to disproportionately few
successful experiences in the replay buffer. Hindsight experience replay (HER)
was recently proposed to tackle this difficulty by manipulating unsuccessful
transitions, but in doing so, HER introduces a significant bias in the replay
buffer experiences and therefore achieves a suboptimal improvement in
sample-efficiency. In this paper, we present an analysis on the source of bias
in HER, and propose a simple and effective method to counter the bias, to most
effectively harness the sample-efficiency provided by HER. Our method,
motivated by counter-factual reasoning and called ARCHER, extends HER with a
trade-off to make rewards calculated for hindsight experiences numerically
greater than real rewards. We validate our algorithm on two continuous control
environments from DeepMind Control Suite - Reacher and Finger, which simulate
manipulation tasks with a robotic arm - in combination with various reward
functions, task complexities and goal sampling strategies. Our experiments
consistently demonstrate that countering bias using more aggressive hindsight
rewards increases sample efficiency, thus establishing the greater benefit of
ARCHER in RL applications with limited computing budget.
Deep learning for in vitro prediction of pharmaceutical formulations
Authors: Yilong Yang, Zhuyifan Ye, Yan Su, Qianqian Zhao, Xiaoshan Li, Defang Ouyang
Date: 2018-09-06 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Prediction

Current pharmaceutical formulation development still strongly relies on the
traditional trial-and-error approach by individual experiences of
pharmaceutical scientists, which is laborious, time-consuming and costly.
Recently, deep learning has been widely applied in many challenging domains
because of its important capability of automatic feature extraction. The aim of
this research is to use deep learning to predict pharmaceutical formulations.
In this paper, two different types of dosage forms were chosen as model
systems. Evaluation criteria suitable for pharmaceutics were applied to
assessing the performance of the models. Moreover, an automatic dataset
selection algorithm was developed for selecting the representative data as
validation and test datasets. Six machine learning methods were compared with
deep learning. The result shows the accuracies of both two deep neural networks
were above 80% and higher than other machine learning models, which showed good
prediction in pharmaceutical formulations. In summary, deep learning with the
automatic data splitting algorithm and the evaluation criteria suitable for
pharmaceutical formulation data was firstly developed for the prediction of
pharmaceutical formulations. The cross-disciplinary integration of
pharmaceutics and artificial intelligence may shift the paradigm of
pharmaceutical researches from experience-dependent studies to data-driven
methodologies.
Two Dimensional Stochastic Configuration Networks for Image Data Analytics
Authors: Ming Li, Dianhui Wang
Date: 2018-09-06 00:00:00
Tasks: Face Recognition, Vocal Bursts Valence Prediction

Stochastic configuration networks (SCNs) as a class of randomized learner
model have been successfully employed in data analytics due to its universal
approximation capability and fast modelling property. The technical essence
lies in stochastically configuring hidden nodes (or basis functions) based on a
supervisory mechanism rather than data-independent randomization as usually
adopted for building randomized neural networks. Given image data modelling
tasks, the use of one-dimensional SCNs potentially demolishes the spatial
information of images, and may result in undesirable performance. This paper
extends the original SCNs to two-dimensional version, termed 2DSCNs, for fast
building randomized learners with matrix-inputs. Some theoretical analyses on
the goodness of 2DSCNs against SCNs, including the complexity of the random
parameter space, and the superiority of generalization, are presented.
Empirical results over one regression, four benchmark handwritten digits
classification, and two human face recognition datasets demonstrate that the
proposed 2DSCNs perform favourably and show good potential for image data
analytics.
Sample-Efficient Imitation Learning via Generative Adversarial Nets
Authors: Lionel Blondé, Alexandros Kalousis
Date: 2018-09-06 00:00:00
Tasks: continuous-control, Continuous Control, Imitation Learning

GAIL is a recent successful imitation learning architecture that exploits the
adversarial training procedure introduced in GANs. Albeit successful at
generating behaviours similar to those demonstrated to the agent, GAIL suffers
from a high sample complexity in the number of interactions it has to carry out
in the environment in order to achieve satisfactory performance. We
dramatically shrink the amount of interactions with the environment necessary
to learn well-behaved imitation policies, by up to several orders of magnitude.
Our framework, operating in the model-free regime, exhibits a significant
increase in sample-efficiency over previous methods by simultaneously a)
learning a self-tuned adversarially-trained surrogate reward and b) leveraging
an off-policy actor-critic architecture. We show that our approach is simple to
implement and that the learned agents remain remarkably stable, as shown in our
experiments that span a variety of continuous control tasks. Video
visualisations available at: \url{https://youtu.be/-nCsqUJnRKU}.
Memory Replay GANs: learning to generate images from new categories without forgetting
Authors: Chenshen Wu, Luis Herranz, Xialei Liu, Yaxing Wang, Joost Van de Weijer, Bogdan Raducanu
Date: 2018-09-06 00:00:00

Previous works on sequential learning address the problem of forgetting in discriminative models. In this paper we consider the case of generative models. In particular, we investigate generative adversarial networks (GANs) in the task of learning new categories in a sequential fashion. We first show that sequential fine tuning renders the network unable to properly generate images from previous categories (i.e. forgetting). Addressing this problem, we propose Memory Replay GANs (MeRGANs), a conditional GAN framework that integrates a memory replay generator. We study two methods to prevent forgetting by leveraging these replays, namely joint training with replay and replay alignment. Qualitative and quantitative experimental results in MNIST, SVHN and LSUN datasets show that our memory replay approach can generate competitive images while significantly mitigating the forgetting of previous categories.
Surface Light Field Fusion
Authors: Jeong Joon Park, Richard Newcombe, Steve Seitz
Date: 2018-09-06 00:00:00

We present an approach for interactively scanning highly reflective objects
with a commodity RGBD sensor. In addition to shape, our approach models the
surface light field, encoding scene appearance from all directions. By
factoring the surface light field into view-independent and
wavelength-independent components, we arrive at a representation that can be
robustly estimated with IR-equipped commodity depth sensors, and achieves high
quality results.
Oblique Stripe Removal in Remote Sensing Images via Oriented Variation
Authors: Xin-Xin Liu, Xiliang Lu, Huanfeng Shen, Qiangqiang Yuan, Liangpei Zhang
Date: 2018-09-06 00:00:00

Destriping is a classical problem in remote sensing image processing.
Although considerable effort has been made to remove stripes, few of the
existing methods can eliminate stripe noise with arbitrary orientations. This
situation makes the removal of oblique stripes in the higher-level remote
sensing products become an unfinished and urgent issue. To overcome the
challenging problem, we propose a novel destriping model which is self-adjusted
to different orientations of stripe noise. First of all, the oriented variation
model is designed to accomplish the stripe orientation approximation. In this
model, the stripe direction is automatically estimated and then imbedded into
the constraint term to depict the along-stripe smoothness of the stripe
component. Mainly based on the oriented variation model, a whole destriping
framework is proposed by jointly employing an L1-norm constraint and a TV
regularization to separately capture the global distribution property of stripe
component and the piecewise smoothness of the clean image. The qualitative and
quantitative experimental results of both orientation and destriping aspects
confirm the effectiveness and stability of the proposed method.