pwc-archive/papers-with-abstracts huggingface dataset:
Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control
Authors: Fabian Ruffy, Michael Przystupa, Ivan Beschastnikh
Date: 2018-12-24 00:00:00
Tasks: Deep Reinforcement Learning, OpenAI Gym, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recent networking research has identified that data-driven congestion control
(CC) can be more efficient than traditional CC in TCP. Deep reinforcement
learning (RL), in particular, has the potential to learn optimal network
policies. However, RL suffers from instability and over-fitting, deficiencies
which so far render it unacceptable for use in datacenter networks. In this
paper, we analyze the requirements for RL to succeed in the datacenter context.
We present a new emulator, Iroko, which we developed to support different
network topologies, congestion control algorithms, and deployment scenarios.
Iroko interfaces with the OpenAI gym toolkit, which allows for fast and fair
evaluation of different RL and traditional CC algorithms under the same
conditions. We present initial benchmarks on three deep RL algorithms compared
to TCP New Vegas and DCTCP. Our results show that these algorithms are able to
learn a CC policy which exceeds the performance of TCP New Vegas on a dumbbell
and fat-tree topology. We make our emulator open-source and publicly available:
https://github.com/dcgym/iroko
VMAV-C: A Deep Attention-based Reinforcement Learning Algorithm for Model-based Control
Authors: Xingxing Liang, Qi. Wang, Yanghe Feng, Zhong Liu, Jincai Huang
Date: 2018-12-24 00:00:00
Tasks: Deep Attention, Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scheduling

Recent breakthroughs in Go play and strategic games have witnessed the great
potential of reinforcement learning in intelligently scheduling in uncertain
environment, but some bottlenecks are also encountered when we generalize this
paradigm to universal complex tasks. Among them, the low efficiency of data
utilization in model-free reinforcement algorithms is of great concern. In
contrast, the model-based reinforcement learning algorithms can reveal
underlying dynamics in learning environments and seldom suffer the data
utilization problem. To address the problem, a model-based reinforcement
learning algorithm with attention mechanism embedded is proposed as an
extension of World Models in this paper. We learn the environment model through
Mixture Density Network Recurrent Network(MDN-RNN) for agents to interact, with
combinations of variational auto-encoder(VAE) and attention incorporated in
state value estimates during the process of learning policy. In this way, agent
can learn optimal policies through less interactions with actual environment,
and final experiments demonstrate the effectiveness of our model in control
problem.
Self-Attention Equipped Graph Convolutions for Disease Prediction
Authors: Anees Kazi, S. Arvind krishna, Shayan Shekarforoush, Karsten Kortuem, Shadi Albarqouni, Nassir Navab
Date: 2018-12-24 00:00:00
Tasks: Disease Prediction, Prediction

Multi-modal data comprising imaging (MRI, fMRI, PET, etc.) and non-imaging
(clinical test, demographics, etc.) data can be collected together and used for
disease prediction. Such diverse data gives complementary information about the
patient\'s condition to make an informed diagnosis. A model capable of
leveraging the individuality of each multi-modal data is required for better
disease prediction. We propose a graph convolution based deep model which takes
into account the distinctiveness of each element of the multi-modal data. We
incorporate a novel self-attention layer, which weights every element of the
demographic data by exploring its relation to the underlying disease. We
demonstrate the superiority of our developed technique in terms of
computational speed and performance when compared to state-of-the-art methods.
Our method outperforms other methods with a significant margin.
A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
Authors: Yang Zhang, Philip David, Hassan Foroosh, Boqing Gong
Date: 2018-12-24 00:00:00
Tasks: Autonomous Driving, Domain Adaptation, Image-to-Image Translation, Segmentation, Semantic Segmentation, Superpixels, Synthetic-to-Real Translation

During the last half decade, convolutional neural networks (CNNs) have
triumphed over semantic segmentation, which is one of the core tasks in many
applications such as autonomous driving and augmented reality. However, to
train CNNs requires a considerable amount of data, which is difficult to
collect and laborious to annotate. Recent advances in computer graphics make it
possible to train CNNs on photo-realistic synthetic imagery with
computer-generated annotations. Despite this, the domain mismatch between the
real images and the synthetic data hinders the models' performance. Hence, we
propose a curriculum-style learning approach to minimizing the domain gap in
urban scene semantic segmentation. The curriculum domain adaptation solves easy
tasks first to infer necessary properties about the target domain; in
particular, the first task is to learn global label distributions over images
and local distributions over landmark superpixels. These are easy to estimate
because images of urban scenes have strong idiosyncrasies (e.g., the size and
spatial relations of buildings, streets, cars, etc.). We then train a
segmentation network, while regularizing its predictions in the target domain
to follow those inferred properties. In experiments, our method outperforms the
baselines on two datasets and two backbone networks. We also report extensive
ablation studies about our approach.
Color Image Enhancement Method Based on Weighted Image Guided Filtering
Authors: Qi Mu, Yanyan Wei, Zhanli Li
Date: 2018-12-24 00:00:00
Tasks: Image Enhancement

A novel color image enhancement method is proposed based on Retinex to
enhance color images under non-uniform illumination or poor visibility
conditions. Different from the conventional Retinex algorithms, the Weighted
Guided Image Filter is used as a surround function instead of the Gaussian
filter to estimate the background illumination, which can overcome the
drawbacks of local blur and halo artifact that may appear by Gaussian filter.
To avoid color distortion, the image is converted to the HSI color model, and
only the intensity channel is enhanced. Then a linear color restoration
algorithm is adopted to convert the enhanced intensity image back to the RGB
color model, which ensures the hue is constant and undistorted. Experimental
results show that the proposed method is effective to enhance both color and
gray images with low exposure and non-uniform illumination, resulting in better
visual quality than traditional method. At the same time, the objective
evaluation indicators are also superior to the conventional methods. In
addition, the efficiency of the proposed method is also improved thanks to the
linear color restoration algorithm.
SNAS: Stochastic Neural Architecture Search
Authors: Sirui Xie, Hehui Zheng, Chunxiao Liu, Liang Lin
Date: 2018-12-24 00:00:00
Proceeding: ICLR 2019 5
Tasks: Neural Architecture Search, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose Stochastic Neural Architecture Search (SNAS), an economical end-to-end solution to Neural Architecture Search (NAS) that trains neural operation parameters and architecture distribution parameters in same round of back-propagation, while maintaining the completeness and differentiability of the NAS pipeline. In this work, NAS is reformulated as an optimization problem on parameters of a joint distribution for the search space in a cell. To leverage the gradient information in generic differentiable loss for architecture search, a novel search gradient is proposed. We prove that this search gradient optimizes the same objective as reinforcement-learning-based NAS, but assigns credits to structural decisions more efficiently. This credit assignment is further augmented with locally decomposable reward to enforce a resource-efficient constraint. In experiments on CIFAR-10, SNAS takes less epochs to find a cell architecture with state-of-the-art accuracy than non-differentiable evolution-based and reinforcement-learning-based NAS, which is also transferable to ImageNet. It is also shown that child networks of SNAS can maintain the validation accuracy in searching, with which attention-based NAS requires parameter retraining to compete, exhibiting potentials to stride towards efficient NAS on big datasets. We have released our implementation at https://github.com/SNAS-Series/SNAS-Series.
Dual Principal Component Pursuit: Probability Analysis and Efficient Algorithms
Authors: Zhihui Zhu, Yifan Wang, Daniel P. Robinson, Daniel Q. Naiman, Rene Vidal, Manolis C. Tsakiris
Date: 2018-12-24 00:00:00

Recent methods for learning a linear subspace from data corrupted by outliers
are based on convex $\ell_1$ and nuclear norm optimization and require the
dimension of the subspace and the number of outliers to be sufficiently small.
In sharp contrast, the recently proposed Dual Principal Component Pursuit
(DPCP) method can provably handle subspaces of high dimension by solving a
non-convex $\ell_1$ optimization problem on the sphere. However, its geometric
analysis is based on quantities that are difficult to interpret and are not
amenable to statistical analysis. In this paper we provide a refined geometric
analysis and a new statistical analysis that show that DPCP can tolerate as
many outliers as the square of the number of inliers, thus improving upon other
provably correct robust PCA methods. We also propose a scalable Projected
Sub-Gradient Method method (DPCP-PSGM) for solving the DPCP problem and show it
admits linear convergence even though the underlying optimization problem is
non-convex and non-smooth. Experiments on road plane detection from 3D point
cloud data demonstrate that DPCP-PSGM can be more efficient than the
traditional RANSAC algorithm, which is one of the most popular methods for such
computer vision applications.
Dynamic Runtime Feature Map Pruning
Authors: Tailin Liang, Lei Wang, Shaobo Shi, John Glossner
Date: 2018-12-24 00:00:00

High bandwidth requirements are an obstacle for accelerating the training and
inference of deep neural networks. Most previous research focuses on reducing
the size of kernel maps for inference. We analyze parameter sparsity of six
popular convolutional neural networks - AlexNet, MobileNet, ResNet-50,
SqueezeNet, TinyNet, and VGG16. Of the networks considered, those using ReLU
(AlexNet, SqueezeNet, VGG16) contain a high percentage of 0-valued parameters
and can be statically pruned. Networks with Non-ReLU activation functions in
some cases may not contain any 0-valued parameters (ResNet-50, TinyNet). We
also investigate runtime feature map usage and find that input feature maps
comprise the majority of bandwidth requirements when depth-wise convolution and
point-wise convolutions used. We introduce dynamic runtime pruning of feature
maps and show that 10% of dynamic feature map execution can be removed without
loss of accuracy. We then extend dynamic pruning to allow for values within an
epsilon of zero and show a further 5% reduction of feature map loading with a
1% loss of accuracy in top-1.
Improving MMD-GAN Training with Repulsive Loss Function
Authors: Wei Wang, Yuan Sun, Saman Halgamuge
Date: 2018-12-24 00:00:00
Proceeding: ICLR 2019 5
Tasks: Image Generation

Generative adversarial nets (GANs) are widely used to learn the data sampling
process and their performance may heavily depend on the loss functions, given a
limited computational budget. This study revisits MMD-GAN that uses the maximum
mean discrepancy (MMD) as the loss function for GAN and makes two
contributions. First, we argue that the existing MMD loss function may
discourage the learning of fine details in data as it attempts to contract the
discriminator outputs of real data. To address this issue, we propose a
repulsive loss function to actively learn the difference among the real data by
simply rearranging the terms in MMD. Second, inspired by the hinge loss, we
propose a bounded Gaussian kernel to stabilize the training of MMD-GAN with the
repulsive loss function. The proposed methods are applied to the unsupervised
image generation tasks on CIFAR-10, STL-10, CelebA, and LSUN bedroom datasets.
Results show that the repulsive loss function significantly improves over the
MMD loss at no additional computational cost and outperforms other
representative loss functions. The proposed methods achieve an FID score of
16.21 on the CIFAR-10 dataset using a single DCGAN network and spectral
normalization.
Image-to-Image Translation via Group-wise Deep Whitening-and-Coloring Transformation
Authors: Wonwoong Cho, Sungha Choi, David Keetae Park, Inkyu Shin, Jaegul Choo
Date: 2018-12-24 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image-to-Image Translation, Style Transfer, Translation

Recently, unsupervised exemplar-based image-to-image translation, conditioned on a given exemplar without the paired data, has accomplished substantial advancements. In order to transfer the information from an exemplar to an input image, existing methods often use a normalization technique, e.g., adaptive instance normalization, that controls the channel-wise statistics of an input activation map at a particular layer, such as the mean and the variance. Meanwhile, style transfer approaches similar task to image translation by nature, demonstrated superior performance by using the higher-order statistics such as covariance among channels in representing a style. In detail, it works via whitening (given a zero-mean input feature, transforming its covariance matrix into the identity). followed by coloring (changing the covariance matrix of the whitened feature to those of the style feature). However, applying this approach in image translation is computationally intensive and error-prone due to the expensive time complexity and its non-trivial backpropagation. In response, this paper proposes an end-to-end approach tailored for image translation that efficiently approximates this transformation with our novel regularization methods. We further extend our approach to a group-wise form for memory and time efficiency as well as image quality. Extensive qualitative and quantitative experiments demonstrate that our proposed method is fast, both in training and inference, and highly effective in reflecting the style of an exemplar. Finally, our code is available at https://github.com/WonwoongCho/GDWCT.
Group Preserving Label Embedding for Multi-Label Classification
Authors: Vikas Kumar, Arun K Pujari, Vineet Padmanabhan, Venkateswara Rao Kagita
Date: 2018-12-24 00:00:00
Tasks: Classification, Dimensionality Reduction, feature selection, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Label Learning

Multi-label learning is concerned with the classification of data with
multiple class labels. This is in contrast to the traditional classification
problem where every data instance has a single label. Due to the exponential
size of output space, exploiting intrinsic information in feature and label
spaces has been the major thrust of research in recent years and use of
parametrization and embedding have been the prime focus. Researchers have
studied several aspects of embedding which include label embedding, input
embedding, dimensionality reduction and feature selection. These approaches
differ from one another in their capability to capture other intrinsic
properties such as label correlation, local invariance etc. We assume here that
the input data form groups and as a result, the label matrix exhibits a
sparsity pattern and hence the labels corresponding to objects in the same
group have similar sparsity. In this paper, we study the embedding of labels
together with the group information with an objective to build an efficient
multi-label classification. We assume the existence of a low-dimensional space
onto which the feature vectors and label vectors can be embedded. In order to
achieve this, we address three sub-problems namely; (1) Identification of
groups of labels; (2) Embedding of label vectors to a low rank-space so that
the sparsity characteristic of individual groups remains invariant; and (3)
Determining a linear mapping that embeds the feature vectors onto the same set
of points, as in stage 2, in the low-dimensional space. We compare our method
with seven well-known algorithms on twelve benchmark data sets. Our
experimental analysis manifests the superiority of our proposed method over
state-of-art algorithms for multi-label learning.
PatientEG Dataset: Bringing Event Graph Model with Temporal Relations to Electronic Medical Records
Authors: Xuli Liu, Jihao Jin, Qi. Wang, Tong Ruan, Yangming Zhou, Daqi Gao, Yichao Yin
Date: 2018-12-24 00:00:00

Medical activities, such as diagnoses, medicine treatments, and laboratory
tests, as well as temporal relations between these activities are the basic
concepts in clinical research. However, existing relational data model on
electronic medical records (EMRs) lacks explicit and accurate semantic
definitions of these concepts. It leads to the inconvenience of query
construction and the inefficiency of query execution where multi-table join
queries are frequently required. In this paper, we propose a patient event
graph (PatientEG) model to capture the characteristics of EMRs. We respectively
define five types of medical entities, five types of medical events and five
types of temporal relations. Based on the proposed model, we also construct a
PatientEG dataset with 191,294 events, 3,429 distinct entities, and 545,993
temporal relations using EMRs from Shanghai Shuguang hospital. To help to
normalize entity values which contain synonyms, hyponymies, and abbreviations,
we link them with the Chinese biomedical knowledge graph. With the help of
PatientEG dataset, we are able to conveniently perform complex queries for
clinical research such as auxiliary diagnosis and therapeutic effectiveness
analysis. In addition, we provide a SPARQL endpoint to access PatientEG dataset
and the dataset is also publicly available online. Also, we list several
illustrative SPARQL queries on our website.
Adaptive Confidence Smoothing for Generalized Zero-Shot Learning
Authors: Yuval Atzmon, Gal Chechik
Date: 2018-12-24 00:00:00
Proceeding: CVPR 2019 6
Tasks: Generalized Zero-Shot Learning, Zero-Shot Learning

Generalized zero-shot learning (GZSL) is the problem of learning a classifier where some classes have samples and others are learned from side information, like semantic attributes or text description, in a zero-shot learning fashion (ZSL). Training a single model that operates in these two regimes simultaneously is challenging. Here we describe a probabilistic approach that breaks the model into three modular components, and then combines them in a consistent way. Specifically, our model consists of three classifiers: A "gating" model that makes soft decisions if a sample is from a "seen" class, and two experts: a ZSL expert, and an expert model for seen classes. We address two main difficulties in this approach: How to provide an accurate estimate of the gating probability without any training samples for unseen classes; and how to use expert predictions when it observes samples outside of its domain. The key insight to our approach is to pass information between the three models to improve each one's accuracy, while maintaining the modular structure. We test our approach, adaptive confidence smoothing (COSMO), on four standard GZSL benchmark datasets and find that it largely outperforms state-of-the-art GZSL models. COSMO is also the first model that closes the gap and surpasses the performance of generative models for GZSL, even-though it is a light-weight model that is much easier to train and tune. Notably, COSMO offers a new view for developing zero-shot models. Thanks to COSMO's modular structure, instead of trying to perform well both on seen and on unseen classes, models can focus on accurate classification of unseen classes, and later consider seen class models.
Invariant and Equivariant Graph Networks
Authors: Haggai Maron, Heli Ben-Hamu, Nadav Shamir, Yaron Lipman
Date: 2018-12-24 00:00:00
Proceeding: ICLR 2019 5
Tasks: 2k

Invariant and equivariant networks have been successfully used for learning
images, sets, point clouds, and graphs. A basic challenge in developing such
networks is finding the maximal collection of invariant and equivariant linear
layers. Although this question is answered for the first three examples (for
popular transformations, at-least), a full characterization of invariant and
equivariant linear layers for graphs is not known.
  In this paper we provide a characterization of all permutation invariant and
equivariant linear layers for (hyper-)graph data, and show that their
dimension, in case of edge-value graph data, is 2 and 15, respectively. More
generally, for graph data defined on k-tuples of nodes, the dimension is the
k-th and 2k-th Bell numbers. Orthogonal bases for the layers are computed,
including generalization to multi-graph data. The constant number of basis
elements and their characteristics allow successfully applying the networks to
different size graphs. From the theoretical point of view, our results
generalize and unify recent advancement in equivariant deep learning. In
particular, we show that our model is capable of approximating any message
passing neural network
  Applying these new linear layers in a simple deep neural network framework is
shown to achieve comparable results to state-of-the-art and to have better
expressivity than previous invariant and equivariant bases.
TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network
Authors: Yipeng Sun, Chengquan Zhang, Zuming Huang, Jiaming Liu, Junyu Han, Errui Ding
Date: 2018-12-24 00:00:00
Tasks: Optical Character Recognition (OCR), Text Detection

Reading text from images remains challenging due to multi-orientation,
perspective distortion and especially the curved nature of irregular text. Most
of existing approaches attempt to solve the problem in two or multiple stages,
which is considered to be the bottleneck to optimize the overall performance.
To address this issue, we propose an end-to-end trainable network architecture,
named TextNet, which is able to simultaneously localize and recognize irregular
text from images. Specifically, we develop a scale-aware attention mechanism to
learn multi-scale image features as a backbone network, sharing fully
convolutional features and computation for localization and recognition. In
text detection branch, we directly generate text proposals in quadrangles,
covering oriented, perspective and curved text regions. To preserve text
features for recognition, we introduce a perspective RoI transform layer, which
can align quadrangle proposals into small feature maps. Furthermore, in order
to extract effective features for recognition, we propose to encode the aligned
RoI features by RNN into context information, combining spatial attention
mechanism to generate text sequences. This overall pipeline is capable of
handling both regular and irregular cases. Finally, text localization and
recognition tasks can be jointly trained in an end-to-end fashion with designed
multi-task loss. Experiments on standard benchmarks show that the proposed
TextNet can achieve state-of-the-art performance, and outperform existing
approaches on irregular datasets by a large margin.