pwc-archive/papers-with-abstracts huggingface dataset:
Digging Deeper into Egocentric Gaze Prediction
Authors: Hamed R. -Tavakoli, Esa Rahtu, Juho Kannala, Ali Borji
Date: 2019-04-12 00:00:00
Tasks: Activity Recognition, Gaze Prediction, Optical Flow Estimation, Prediction, Transfer Learning

This paper digs deeper into factors that influence egocentric gaze. Instead
of training deep models for this purpose in a blind manner, we propose to
inspect factors that contribute to gaze guidance during daily tasks. Bottom-up
saliency and optical flow are assessed versus strong spatial prior baselines.
Task-specific cues such as vanishing point, manipulation point, and hand
regions are analyzed as representatives of top-down information. We also look
into the contribution of these factors by investigating a simple recurrent
neural model for ego-centric gaze prediction. First, deep features are
extracted for all input video frames. Then, a gated recurrent unit is employed
to integrate information over time and to predict the next fixation. We also
propose an integrated model that combines the recurrent model with several
top-down and bottom-up cues. Extensive experiments over multiple datasets
reveal that (1) spatial biases are strong in egocentric videos, (2) bottom-up
saliency models perform poorly in predicting gaze and underperform spatial
biases, (3) deep features perform better compared to traditional features, (4)
as opposed to hand regions, the manipulation point is a strong influential cue
for gaze prediction, (5) combining the proposed recurrent model with bottom-up
cues, vanishing points and, in particular, manipulation point results in the
best gaze prediction accuracy over egocentric videos, (6) the knowledge
transfer works best for cases where the tasks or sequences are similar, and (7)
task and activity recognition can benefit from gaze prediction. Our findings
suggest that (1) there should be more emphasis on hand-object interaction and
(2) the egocentric vision community should consider larger datasets including
diverse stimuli and more subjects.
Unsupervised Speech Domain Adaptation Based on Disentangled Representation Learning for Robust Speech Recognition
Authors: Jong-Hyeon Park, Myungwoo Oh, Hyung-Min Park
Date: 2019-04-12 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Domain Adaptation, Image-to-Image Translation, Representation Learning, Robust Speech Recognition, speech-recognition, Speech Recognition, Translation

In general, the performance of automatic speech recognition (ASR) systems is
significantly degraded due to the mismatch between training and test
environments. Recently, a deep-learning-based image-to-image translation
technique to translate an image from a source domain to a desired domain was
presented, and cycle-consistent adversarial network (CycleGAN) was applied to
learn a mapping for speech-to-speech conversion from a speaker to a target
speaker. However, this method might not be adequate to remove corrupting noise
components for robust ASR because it was designed to convert speech itself. In
this paper, we propose a domain adaptation method based on generative
adversarial nets (GANs) with disentangled representation learning to achieve
robustness in ASR systems. In particular, two separated encoders, context and
domain encoders, are introduced to learn distinct latent variables. The latent
variables allow us to convert the domain of speech according to its context and
domain representation. We improved word accuracies by 6.55~15.70\% for the
CHiME4 challenge corpus by applying a noisy-to-clean environment adaptation for
robust ASR. In addition, similar to the method based on the CycleGAN, this
method can be used for gender adaptation in gender-mismatched recognition.
Multi-View Region Adaptive Multi-temporal DMM and RGB Action Recognition
Authors: Mahmoud Al-Faris, John P. Chiverton, Yanyan Yang, David L. Ndzi
Date: 2019-04-12 00:00:00
Tasks: Action Recognition, Human-Object Interaction Detection, Temporal Action Localization

Human action recognition remains an important yet challenging task. This work
proposes a novel action recognition system. It uses a novel Multiple View
Region Adaptive Multi-resolution in time Depth Motion Map (MV-RAMDMM)
formulation combined with appearance information. Multiple stream 3D
Convolutional Neural Networks (CNNs) are trained on the different views and
time resolutions of the region adaptive Depth Motion Maps. Multiple views are
synthesised to enhance the view invariance. The region adaptive weights, based
on localised motion, accentuate and differentiate parts of actions possessing
faster motion. Dedicated 3D CNN streams for multi-time resolution appearance
information (RGB) are also included. These help to identify and differentiate
between small object interactions. A pre-trained 3D-CNN is used here with
fine-tuning for each stream along with multiple class Support Vector Machines
(SVM)s. Average score fusion is used on the output. The developed approach is
capable of recognising both human action and human-object interaction. Three
public domain datasets including: MSR 3D Action,Northwestern UCLA multi-view
actions and MSR 3D daily activity are used to evaluate the proposed solution.
The experimental results demonstrate the robustness of this approach compared
with state-of-the-art algorithms.
AI-IMU Dead-Reckoning
Authors: Martin Brossard, Axel Barrau, Silvère Bonnabel
Date: 2019-04-12 00:00:00
Tasks: Dead-Reckoning Prediction, Navigate

In this paper we propose a novel accurate method for dead-reckoning of
wheeled vehicles based only on an Inertial Measurement Unit (IMU). In the
context of intelligent vehicles, robust and accurate dead-reckoning based on
the IMU may prove useful to correlate feeds from imaging sensors, to safely
navigate through obstructions, or for safe emergency stops in the extreme case
of exteroceptive sensors failure. The key components of the method are the
Kalman filter and the use of deep neural networks to dynamically adapt the
noise parameters of the filter. The method is tested on the KITTI odometry
dataset, and our dead-reckoning inertial method based only on the IMU
accurately estimates 3D position, velocity, orientation of the vehicle and
self-calibrates the IMU biases. We achieve on average a 1.10% translational
error and the algorithm competes with top-ranked methods which, by contrast,
use LiDAR or stereo vision. We make our implementation open-source at:
https://github.com/mbrossar/ai-imu-dr
Building a mixed-lingual neural TTS system with only monolingual data
Authors: Liumeng Xue, Wei Song, Guanghui Xu, Lei Xie, Zhizheng Wu
Date: 2019-04-12 00:00:00
Tasks: Decoder, text-to-speech, Text to Speech

When deploying a Chinese neural text-to-speech (TTS) synthesis system, one of the challenges is to synthesize Chinese utterances with English phrases or words embedded. This paper looks into the problem in the encoder-decoder framework when only monolingual data from a target speaker is available. Specifically, we view the problem from two aspects: speaker consistency within an utterance and naturalness. We start the investigation with an Average Voice Model which is built from multi-speaker monolingual data, i.e. Mandarin and English data. On the basis of that, we look into speaker embedding for speaker consistency within an utterance and phoneme embedding for naturalness and intelligibility and study the choice of data for model training. We report the findings and discuss the challenges to build a mixed-lingual TTS system with only monolingual data.
Unifying Heterogeneous Classifiers with Distillation
Authors: Jayakorn Vongkulbhisal, Phongtharin Vinayavekhin, Marco Visentini-Scarzanella
Date: 2019-04-12 00:00:00
Proceeding: CVPR 2019 6
Tasks: Knowledge Distillation

In this paper, we study the problem of unifying knowledge from a set of
classifiers with different architectures and target classes into a single
classifier, given only a generic set of unlabelled data. We call this problem
Unifying Heterogeneous Classifiers (UHC). This problem is motivated by
scenarios where data is collected from multiple sources, but the sources cannot
share their data, e.g., due to privacy concerns, and only privately trained
models can be shared. In addition, each source may not be able to gather data
to train all classes due to data availability at each source, and may not be
able to train the same classification model due to different computational
resources. To tackle this problem, we propose a generalisation of knowledge
distillation to merge HCs. We derive a probabilistic relation between the
outputs of HCs and the probability over all classes. Based on this relation, we
propose two classes of methods based on cross-entropy minimisation and matrix
factorisation, which allow us to estimate soft labels over all classes from
unlabelled samples and use them in lieu of ground truth labels to train a
unified classifier. Our extensive experiments on ImageNet, LSUN, and Places365
datasets show that our approaches significantly outperform a naive extension of
distillation and can achieve almost the same accuracy as classifiers that are
trained in a centralised, supervised manner.
Distributed Layer-Partitioned Training for Privacy-Preserved Deep Learning
Authors: Chun-Hsien Yu, Chun-Nan Chou, Emily Chang
Date: 2019-04-12 00:00:00
Tasks: Deep Learning, Privacy Preserving, Privacy Preserving Deep Learning

Deep Learning techniques have achieved remarkable results in many domains.
Often, training deep learning models requires large datasets, which may require
sensitive information to be uploaded to the cloud to accelerate training. To
adequately protect sensitive information, we propose distributed
layer-partitioned training with step-wise activation functions for
privacy-preserving deep learning. Experimental results attest our method to be
simple and effective.
Unsupervised Method to Localize Masses in Mammograms
Authors: Bilal Ahmed Lodhi
Date: 2019-04-12 00:00:00
Tasks: Clustering

Breast cancer is one of the most common and prevalent type of cancer that
mainly affects the women population. chances of effective treatment increases
with early diagnosis. Mammography is considered one of the effective and proven
techniques for early diagnosis of breast cancer. Tissues around masses look
identical in mammogram, which makes automatic detection process a very
challenging task. They are indistinguishable from the surrounding parenchyma.
In this paper, we present an efficient and automated approach to segment masses
in mammograms. The proposed method uses hierarchical clustering to isolate the
salient area, and then features are extracted to reject false detection. We
applied our method on two popular publicly available datasets (mini-MIAS and
DDSM). A total of 56 images from mini-mias database, and 76 images from DDSM
were randomly selected. Results are explained in-terms of ROC (Receiver
Operating Characteristics) curves and compared with the other techniques.
Experimental results demonstrate the efficiency and advantages of the proposed
system in automatic mass identification in mammograms.
Adaptive Weighting Multi-Field-of-View CNN for Semantic Segmentation in Pathology
Authors: Hiroki Tokunaga, Yuki Teramoto, Akihiko Yoshizawa, Ryoma Bise
Date: 2019-04-12 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Segmentation, Segmentation, Semantic Segmentation, whole slide images

Automated digital histopathology image segmentation is an important task to
help pathologists diagnose tumors and cancer subtypes. For pathological
diagnosis of cancer subtypes, pathologists usually change the magnification of
whole-slide images (WSI) viewers. A key assumption is that the importance of
the magnifications depends on the characteristics of the input image, such as
cancer subtypes. In this paper, we propose a novel semantic segmentation
method, called Adaptive-Weighting-Multi-Field-of-View-CNN (AWMF-CNN), that can
adaptively use image features from images with different magnifications to
segment multiple cancer subtype regions in the input image. The proposed method
aggregates several expert CNNs for images of different magnifications by
adaptively changing the weight of each expert depending on the input image. It
leverages information in the images with different magnifications that might be
useful for identifying the subtypes. It outperformed other state-of-the-art
methods in experiments.
Effective Scheduling Function Design in SDN through Deep Reinforcement Learning
Authors: Huang Victoria, Chen Gang, Fu Qiang
Date: 2019-04-12 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scheduling

Recent research on Software-Defined Networking (SDN) strongly promotes the
adoption of distributed controller architectures. To achieve high network
performance, designing a scheduling function (SF) to properly dispatch requests
from each switch to suitable controllers becomes critical. However, existing
literature tends to design the SF targeted at specific network settings. In
this paper, a reinforcement-learning-based (RL) approach is proposed with the
aim to automatically learn a general, effective, and efficient SF. In
particular, a new dispatching system is introduced in which the SF is
represented as a neural network that determines the priority of each
controller. Based on the priorities, a controller is selected using our
proposed probability selection scheme to balance the trade-off between
exploration and exploitation during learning. In order to train a general SF,
we first formulate the scheduling function design problem as an RL problem.
Then a new training approach is developed based on a state-of-the-art deep RL
algorithm. Our simulation results show that our RL approach can rapidly design
(or learn) SFs with optimal performance. Apart from that, the trained SF can
generalize well and outperforms commonly used scheduling heuristics under
various network settings.
Evaluating the Representational Hub of Language and Vision Models
Authors: Ravi Shekhar, Ece Takmaz, Raquel Fernández, Raffaella Bernardi
Date: 2019-04-12 00:00:00
Proceeding: WS 2019 5
Tasks: Diagnostic, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

The multimodal models used in the emerging field at the intersection of
computational linguistics and computer vision implement the bottom-up
processing of the `Hub and Spoke' architecture proposed in cognitive science to
represent how the brain processes and combines multi-sensory inputs. In
particular, the Hub is implemented as a neural network encoder. We investigate
the effect on this encoder of various vision-and-language tasks proposed in the
literature: visual question answering, visual reference resolution, and
visually grounded dialogue. To measure the quality of the representations
learned by the encoder, we use two kinds of analyses. First, we evaluate the
encoder pre-trained on the different vision-and-language tasks on an existing
diagnostic task designed to assess multimodal semantic understanding. Second,
we carry out a battery of analyses aimed at studying how the encoder merges and
exploits the two modalities.
Direct speech-to-speech translation with a sequence-to-sequence model
Authors: Ye Jia, Ron J. Weiss, Fadi Biadsy, Wolfgang Macherey, Melvin Johnson, Zhifeng Chen, Yonghui Wu
Date: 2019-04-12 00:00:00
Tasks: Speech Synthesis, Speech-to-Speech Translation, Speech-to-Text, Speech-to-Text Translation, text-to-speech, Text to Speech, Text-To-Speech Synthesis, Translation

We present an attention-based sequence-to-sequence neural network which can directly translate speech from one language into speech in another language, without relying on an intermediate text representation. The network is trained end-to-end, learning to map speech spectrograms into target spectrograms in another language, corresponding to the translated content (in a different canonical voice). We further demonstrate the ability to synthesize translated speech using the voice of the source speaker. We conduct experiments on two Spanish-to-English speech translation datasets, and find that the proposed model slightly underperforms a baseline cascade of a direct speech-to-text translation model and a text-to-speech synthesis model, demonstrating the feasibility of the approach on this very challenging task.
Supervised Anomaly Detection based on Deep Autoregressive Density Estimators
Authors: Tomoharu Iwata, Yuki Yamanaka
Date: 2019-04-12 00:00:00
Tasks: Anomaly Detection, Density Estimation, Supervised Anomaly Detection, Unsupervised Anomaly Detection

We propose a supervised anomaly detection method based on neural density
estimators, where the negative log likelihood is used for the anomaly score.
Density estimators have been widely used for unsupervised anomaly detection. By
the recent advance of deep learning, the density estimation performance has
been greatly improved. However, the neural density estimators cannot exploit
anomaly label information, which would be valuable for improving the anomaly
detection performance. The proposed method effectively utilizes the anomaly
label information by training the neural density estimator so that the
likelihood of normal instances is maximized and the likelihood of anomalous
instances is lower than that of the normal instances. We employ an
autoregressive model for the neural density estimator, which enables us to
calculate the likelihood exactly. With the experiments using 16 datasets, we
demonstrate that the proposed method improves the anomaly detection performance
with a few labeled anomalous instances, and achieves better performance than
existing unsupervised and supervised anomaly detection methods.
EvalNorm: Estimating Batch Normalization Statistics for Evaluation
Authors: Saurabh Singh, Abhinav Shrivastava
Date: 2019-04-12 00:00:00
Proceeding: ICCV 2019 10
Tasks: object-detection, Object Detection

Batch normalization (BN) has been very effective for deep learning and is widely used. However, when training with small minibatches, models using BN exhibit a significant degradation in performance. In this paper we study this peculiar behavior of BN to gain a better understanding of the problem, and identify a cause. We propose 'EvalNorm' to address the issue by estimating corrected normalization statistics to use for BN during evaluation. EvalNorm supports online estimation of the corrected statistics while the model is being trained, and does not affect the training scheme of the model. As a result, EvalNorm can also be used with existing pre-trained models allowing them to benefit from our method. EvalNorm yields large gains for models trained with smaller batches. Our experiments show that EvalNorm performs 6.18% (absolute) better than vanilla BN for a batchsize of 2 on ImageNet validation set and from 1.5 to 7.0 points (absolute) gain on the COCO object detection benchmark across a variety of setups.
Cycle-Consistent Adversarial GAN: the integration of adversarial attack and defense
Authors: Lingyun Jiang, Kai Qiao, Ruoxi Qin, Linyuan Wang, Jian Chen, Haibing Bu, Bin Yan
Date: 2019-04-12 00:00:00
Tasks: Adversarial Attack, image-classification, Image Classification

In image classification of deep learning, adversarial examples where inputs
intended to add small magnitude perturbations may mislead deep neural networks
(DNNs) to incorrect results, which means DNNs are vulnerable to them. Different
attack and defense strategies have been proposed to better research the
mechanism of deep learning. However, those research in these networks are only
for one aspect, either an attack or a defense, not considering that attacks and
defenses should be interdependent and mutually reinforcing, just like the
relationship between spears and shields. In this paper, we propose
Cycle-Consistent Adversarial GAN (CycleAdvGAN) to generate adversarial
examples, which can learn and approximate the distribution of original
instances and adversarial examples. For CycleAdvGAN, once the Generator and are
trained, can generate adversarial perturbations efficiently for any instance,
so as to make DNNs predict wrong, and recovery adversarial examples to clean
instances, so as to make DNNs predict correct. We apply CycleAdvGAN under
semi-white box and black-box settings on two public datasets MNIST and CIFAR10.
Using the extensive experiments, we show that our method has achieved the
state-of-the-art adversarial attack method and also efficiently improve the
defense ability, which make the integration of adversarial attack and defense
come true. In additional, it has improved attack effect only trained on the
adversarial dataset generated by any kind of adversarial attack.
Interaction-aware Decision Making with Adaptive Strategies under Merging Scenarios
Authors: Yeping Hu, Alireza Nakhaei, Masayoshi Tomizuka, Kikuo Fujimura
Date: 2019-04-12 00:00:00
Tasks: Autonomous Vehicles, Common Sense Reasoning, Decision Making, Multi-agent Reinforcement Learning, Reinforcement Learning

In order to drive safely and efficiently under merging scenarios, autonomous vehicles should be aware of their surroundings and make decisions by interacting with other road participants. Moreover, different strategies should be made when the autonomous vehicle is interacting with drivers having different level of cooperativeness. Whether the vehicle is on the merge-lane or main-lane will also influence the driving maneuvers since drivers will behave differently when they have the right-of-way than otherwise. Many traditional methods have been proposed to solve decision making problems under merging scenarios. However, these works either are incapable of modeling complicated interactions or require implementing hand-designed rules which cannot properly handle the uncertainties in real-world scenarios. In this paper, we proposed an interaction-aware decision making with adaptive strategies (IDAS) approach that can let the autonomous vehicle negotiate the road with other drivers by leveraging their cooperativeness under merging scenarios. A single policy is learned under the multi-agent reinforcement learning (MARL) setting via the curriculum learning strategy, which enables the agent to automatically infer other drivers' various behaviors and make decisions strategically. A masking mechanism is also proposed to prevent the agent from exploring states that violate common sense of human judgment and increase the learning efficiency. An exemplar merging scenario was used to implement and examine the proposed method.
A Light Dual-Task Neural Network for Haze Removal
Authors: Yu Zhang, Xinchao Wang, Xiaojun Bi, DaCheng Tao
Date: 2019-04-12 00:00:00
Tasks: Image Dehazing, Single Image Dehazing

Single-image dehazing is a challenging problem due to its ill-posed nature.
Existing methods rely on a suboptimal two-step approach, where an intermediate
product like a depth map is estimated, based on which the haze-free image is
subsequently generated using an artificial prior formula. In this paper, we
propose a light dual-task Neural Network called LDTNet that restores the
haze-free image in one shot. We use transmission map estimation as an auxiliary
task to assist the main task, haze removal, in feature extraction and to
enhance the generalization of the network. In LDTNet, the haze-free image and
the transmission map are produced simultaneously. As a result, the artificial
prior is reduced to the smallest extent. Extensive experiments demonstrate that
our algorithm achieves superior performance against the state-of-the-art
methods on both synthetic and real-world images.