pwc-archive/papers-with-abstracts huggingface dataset:
Attend More Times for Image Captioning
Authors: Jiajun Du, Yu Qin, Hongtao Lu, Yonghua Zhang
Date: 2018-12-08 00:00:00
Tasks: Image Captioning

Most attention-based image captioning models attend to the image once per
word. However, attending once per word is rigid and is easy to miss some
information. Attending more times can adjust the attention position, find the
missing information back and avoid generating the wrong word. In this paper, we
show that attending more times per word can gain improvements in the image
captioning task, without increasing the number of parameters. We propose a
flexible two-LSTM merge model to make it convenient to encode more attentions
than words. Our captioning model uses two LSTMs to encode the word sequence and
the attention sequence respectively. The information of the two LSTMs and the
image feature are combined to predict the next word. Experiments on the MSCOCO
caption dataset show that our method outperforms the state-of-the-art. Using
bottom up features and self-critical training method, our method gets BLEU-4,
METEOR, ROUGE-L, CIDEr and SPICE scores of 0.381, 0.283, 0.580, 1.261 and 0.220
on the Karpathy test split.
Spatial-Temporal Person Re-identification
Authors: Guangcong Wang, Jian-Huang Lai, Peigen Huang, Xiaohua Xie
Date: 2018-12-08 00:00:00
Tasks: Person Re-Identification

Most of current person re-identification (ReID) methods neglect a
spatial-temporal constraint. Given a query image, conventional methods compute
the feature distances between the query image and all the gallery images and
return a similarity ranked table. When the gallery database is very large in
practice, these approaches fail to obtain a good performance due to appearance
ambiguity across different camera views. In this paper, we propose a novel
two-stream spatial-temporal person ReID (st-ReID) framework that mines both
visual semantic information and spatial-temporal information. To this end, a
joint similarity metric with Logistic Smoothing (LS) is introduced to integrate
two kinds of heterogeneous information into a unified framework. To approximate
a complex spatial-temporal probability distribution, we develop a fast
Histogram-Parzen (HP) method. With the help of the spatial-temporal constraint,
the st-ReID model eliminates lots of irrelevant images and thus narrows the
gallery database. Without bells and whistles, our st-ReID method achieves
rank-1 accuracy of 98.1\% on Market-1501 and 94.4\% on DukeMTMC-reID, improving
from the baselines 91.2\% and 83.8\%, respectively, outperforming all previous
state-of-the-art methods by a large margin.
SANTIS: Sampling-Augmented Neural neTwork with Incoherent Structure for MR image reconstruction
Authors: Fang Liu, Lihua Chen, Richard Kijowski, Li Feng
Date: 2018-12-08 00:00:00
Tasks: Deep Learning, Image Reconstruction, Open-Ended Question Answering

Deep learning holds great promise in the reconstruction of undersampled
Magnetic Resonance Imaging (MRI) data, providing new opportunities to escalate
the performance of rapid MRI. In existing deep learning-based reconstruction
methods, supervised training is performed using artifact-free reference images
and their corresponding undersampled pairs. The undersampled images are
generated by a fixed undersampling pattern in the training, and the trained
network is then applied to reconstruct new images acquired with the same
pattern in the inference. While such a training strategy can maintain a
favorable reconstruction for a pre-selected undersampling pattern, the
robustness of the trained network against any discrepancy of undersampling
schemes is typically poor. We developed a novel deep learning-based
reconstruction framework called SANTIS for efficient MR image reconstruction
with improved robustness against sampling pattern discrepancy. SANTIS uses a
data cycle-consistent adversarial network combining efficient end-to-end
convolutional neural network mapping, data fidelity enforcement and adversarial
training for reconstructing accelerated MR images more faithfully. A training
strategy employing sampling augmentation with extensive variation of
undersampling patterns was further introduced to promote the robustness of the
trained network. Compared to conventional reconstruction and standard deep
learning methods, SANTIS achieved consistent better reconstruction performance,
with lower errors, greater image sharpness and higher similarity with respect
to the reference regardless of the undersampling patterns during inference.
This novel concept behind SANTIS can particularly be useful towards improving
the robustness of deep learning-based image reconstruction against discrepancy
between training and evaluation, which is currently an important but less
studied open question.
Generalized Batch Normalization: Towards Accelerating Deep Neural Networks
Authors: Xiaoyong Yuan, Zheng Feng, Matthew Norton, Xiaolin Li
Date: 2018-12-08 00:00:00
Tasks: Management

Utilizing recently introduced concepts from statistics and quantitative risk
management, we present a general variant of Batch Normalization (BN) that
offers accelerated convergence of Neural Network training compared to
conventional BN. In general, we show that mean and standard deviation are not
always the most appropriate choice for the centering and scaling procedure
within the BN transformation, particularly if ReLU follows the normalization
step. We present a Generalized Batch Normalization (GBN) transformation, which
can utilize a variety of alternative deviation measures for scaling and
statistics for centering, choices which naturally arise from the theory of
generalized deviation measures and risk theory in general. When used in
conjunction with the ReLU non-linearity, the underlying risk theory suggests
natural, arguably optimal choices for the deviation measure and statistic.
Utilizing the suggested deviation measure and statistic, we show experimentally
that training is accelerated more so than with conventional BN, often with
improved error rate as well. Overall, we propose a more flexible BN
transformation supported by a complimentary theoretical framework that can
potentially guide design choices.
Neural Abstract Style Transfer for Chinese Traditional Painting
Authors: Bo Li, Caiming Xiong, Tianfu Wu, Yu Zhou, Lun Zhang, Rufeng Chu
Date: 2018-12-08 00:00:00
Tasks: Style Transfer

Chinese traditional painting is one of the most historical artworks in the
world. It is very popular in Eastern and Southeast Asia due to being
aesthetically appealing. Compared with western artistic painting, it is usually
more visually abstract and textureless. Recently, neural network based style
transfer methods have shown promising and appealing results which are mainly
focused on western painting. It remains a challenging problem to preserve
abstraction in neural style transfer. In this paper, we present a Neural
Abstract Style Transfer method for Chinese traditional painting. It learns to
preserve abstraction and other style jointly end-to-end via a novel
MXDoG-guided filter (Modified version of the eXtended Difference-of-Gaussians)
and three fully differentiable loss terms. To the best of our knowledge, there
is little work study on neural style transfer of Chinese traditional painting.
To promote research on this direction, we collect a new dataset with diverse
photo-realistic images and Chinese traditional paintings. In experiments, the
proposed method shows more appealing stylized results in transferring the style
of Chinese traditional painting than state-of-the-art neural style transfer
methods.
An Exploratory Study of (#)Exercise in the Twittersphere
Authors: George Shaw, Amir Karami
Date: 2018-12-08 00:00:00

Social media analytics allows us to extract, analyze, and establish semantic
from user-generated contents in social media platforms. This study utilized a
mixed method including a three-step process of data collection, topic modeling,
and data annotation for recognizing exercise related patterns. Based on the
findings, 86% of the detected topics were identified as meaningful topics after
conducting the data annotation process. The most discussed exercise-related
topics were physical activity (18.7%), lifestyle behaviors (6.6%), and dieting
(4%). The results from our experiment indicate that the exploratory data
analysis is a practical approach to summarizing the various characteristics of
text data for different health and medical applications.
Political Popularity Analysis in Social Media
Authors: Amir Karami, Aida Elkouri
Date: 2018-12-08 00:00:00

Popularity is a critical success factor for a politician and her/his party to
win in elections and implement their plans. Finding the reasons behind the
popularity can provide a stable political movement. This research attempts to
measure popularity in Twitter using a mixed method. In recent years, Twitter
data has provided an excellent opportunity for exploring public opinions by
analyzing a large number of tweets. This study has collected and examined 4.5
million tweets related to a US politician, Senator Bernie Sanders. This study
investigated eight economic reasons behind the senator's popularity in Twitter.
This research has benefits for politicians, informatics experts, and
policymakers to explore public opinion. The collected data will also be
available for further investigation.
Counterfactuals uncover the modular structure of deep generative models
Authors: Michel Besserve, Arash Mehrjou, Rémy Sun, Bernhard Schölkopf
Date: 2018-12-08 00:00:00
Proceeding: ICLR 2020 1
Tasks: counterfactual, Style Transfer

Deep generative models can emulate the perceptual properties of complex image datasets, providing a latent representation of the data. However, manipulating such representation to perform meaningful and controllable transformations in the data space remains challenging without some form of supervision. While previous work has focused on exploiting statistical independence to disentangle latent factors, we argue that such requirement is too restrictive and propose instead a non-statistical framework that relies on counterfactual manipulations to uncover a modular structure of the network composed of disentangled groups of internal variables. Experiments with a variety of generative models trained on complex image datasets show the obtained modules can be used to design targeted interventions. This opens the way to applications such as computationally efficient style transfer and the automated assessment of robustness to contextual changes in pattern recognition systems.
Face Completion with Semantic Knowledge and Collaborative Adversarial Learning
Authors: Haofu Liao, Gareth Funka-Lea, Yefeng Zheng, Jiebo Luo, S. Kevin Zhou
Date: 2018-12-08 00:00:00
Tasks: Facial Inpainting, Image Inpainting, Semantic Segmentation

Unlike a conventional background inpainting approach that infers a missing area from image patches similar to the background, face completion requires semantic knowledge about the target object for realistic outputs. Current image inpainting approaches utilize generative adversarial networks (GANs) to achieve such semantic understanding. However, in adversarial learning, the semantic knowledge is learned implicitly and hence good semantic understanding is not always guaranteed. In this work, we propose a collaborative adversarial learning approach to face completion to explicitly induce the training process. Our method is formulated under a novel generative framework called collaborative GAN (collaGAN), which allows better semantic understanding of a target object through collaborative learning of multiple tasks including face completion, landmark detection, and semantic segmentation. Together with the collaGAN, we also introduce an inpainting concentrated scheme such that the model emphasizes more on inpainting instead of autoencoding. Extensive experiments show that the proposed designs are indeed effective and collaborative adversarial learning provides better feature representations of the faces. In comparison with other generative image inpainting models and single task learning methods, our solution produces superior performances on all tasks.
Deep ChArUco: Dark ChArUco Marker Pose Estimation
Authors: Danying Hu, Daniel DeTone, Vikram Chauhan, Igor Spivak, Tomasz Malisiewicz
Date: 2018-12-08 00:00:00
Proceeding: CVPR 2019 6
Tasks: Camera Calibration, Data Augmentation, Pose Estimation

ChArUco boards are used for camera calibration, monocular pose estimation, and pose verification in both robotics and augmented reality. Such fiducials are detectable via traditional computer vision methods (as found in OpenCV) in well-lit environments, but classical methods fail when the lighting is poor or when the image undergoes extreme motion blur. We present Deep ChArUco, a real-time pose estimation system which combines two custom deep networks, ChArUcoNet and RefineNet, with the Perspective-n-Point (PnP) algorithm to estimate the marker's 6DoF pose. ChArUcoNet is a two-headed marker-specific convolutional neural network (CNN) which jointly outputs ID-specific classifiers and 2D point locations. The 2D point locations are further refined into subpixel coordinates using RefineNet. Our networks are trained using a combination of auto-labeled videos of the target marker, synthetic subpixel corner data, and extreme data augmentation. We evaluate Deep ChArUco in challenging low-light, high-motion, high-blur scenarios and demonstrate that our approach is superior to a traditional OpenCV-based method for ChArUco marker detection and pose estimation.
Self-Improving Visual Odometry
Authors: Daniel DeTone, Tomasz Malisiewicz, Andrew Rabinovich
Date: 2018-12-08 00:00:00
Tasks: 2D Pose Estimation, Pose Estimation, Self-Supervised Learning, Visual Odometry

We propose a self-supervised learning framework that uses unlabeled monocular
video sequences to generate large-scale supervision for training a Visual
Odometry (VO) frontend, a network which computes pointwise data associations
across images. Our self-improving method enables a VO frontend to learn over
time, unlike other VO and SLAM systems which require time-consuming hand-tuning
or expensive data collection to adapt to new environments. Our proposed
frontend operates on monocular images and consists of a single multi-task
convolutional neural network which outputs 2D keypoints locations, keypoint
descriptors, and a novel point stability score. We use the output of VO to
create a self-supervised dataset of point correspondences to retrain the
frontend. When trained using VO at scale on 2.5 million monocular images from
ScanNet, the stability classifier automatically discovers a ranking for
keypoints that are not likely to help in VO, such as t-junctions across depth
discontinuities, features on shadows and highlights, and dynamic objects like
people. The resulting frontend outperforms both traditional methods (SIFT, ORB,
AKAZE) and deep learning methods (SuperPoint and LF-Net) in a 3D-to-2D pose
estimation task on ScanNet.
Efficient Concept Induction for Description Logics
Authors: Md. Kamruzzaman Sarker, Pascal Hitzler
Date: 2018-12-08 00:00:00

Concept Induction refers to the problem of creating complex Description Logic
class descriptions (i.e., TBox axioms) from instance examples (i.e., ABox
data). In this paper we look particularly at the case where both a set of
positive and a set of negative instances are given, and complex class
expressions are sought under which the positive but not the negative examples
fall. Concept induction has found applications in ontology engineering, but
existing algorithms have fundamental performance issues in some scenarios,
mainly because a high number of invokations of an external Description Logic
reasoner is usually required. In this paper we present a new algorithm for this
problem which drastically reduces the number of reasoner invokations needed.
While this comes at the expense of a more limited traversal of the search
space, we show that our approach improves execution times by up to several
orders of magnitude, while output correctness, measured in the amount of
correct coverage of the input instances, remains reasonably high in many cases.
Our approach thus should provide a strong alternative to existing systems, in
particular in settings where other systems are prohibitively slow.
Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning
Authors: Tianyi Chen, Kaiqing Zhang, Georgios B. Giannakis, Tamer Başar
Date: 2018-12-07 00:00:00
Tasks: Distributed Computing, Multi-agent Reinforcement Learning, Policy Gradient Methods, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper deals with distributed policy optimization in reinforcement learning, which involves a central controller and a group of learners. In particular, two typical settings encountered in several applications are considered: multi-agent reinforcement learning (RL) and parallel RL, where frequent information exchanges between the learners and the controller are required. For many practical distributed systems, however, the overhead caused by these frequent communication exchanges is considerable, and becomes the bottleneck of the overall performance. To address this challenge, a novel policy gradient approach is developed for solving distributed RL. The novel approach adaptively skips the policy gradient communication during iterations, and can reduce the communication overhead without degrading learning performance. It is established analytically that: i) the novel algorithm has convergence rate identical to that of the plain-vanilla policy gradient; while ii) if the distributed learners are heterogeneous in terms of their reward functions, the number of communication rounds needed to achieve a desirable learning accuracy is markedly reduced. Numerical experiments corroborate the communication reduction attained by the novel algorithm compared to alternatives.
Improved Knowledge Graph Embedding using Background Taxonomic Information
Authors: Bahare Fatemi, Siamak Ravanbakhsh, David Poole
Date: 2018-12-07 00:00:00
Tasks: Graph Embedding, Knowledge Graph Completion, Knowledge Graph Embedding, Knowledge Graphs

Knowledge graphs are used to represent relational information in terms of
triples. To enable learning about domains, embedding models, such as tensor
factorization models, can be used to make predictions of new triples. Often
there is background taxonomic information (in terms of subclasses and
subproperties) that should also be taken into account. We show that existing
fully expressive (a.k.a. universal) models cannot provably respect subclass and
subproperty information. We show that minimal modifications to an existing
knowledge graph completion method enables injection of taxonomic information.
Moreover, we prove that our model is fully expressive, assuming a lower-bound
on the size of the embeddings. Experimental results on public knowledge graphs
show that despite its simplicity our approach is surprisingly effective.
Reaching Data Confidentiality and Model Accountability on the CalTrain
Authors: Zhongshu Gu, Hani Jamjoom, Dong Su, Heqing Huang, Jialong Zhang, Tengfei Ma, Dimitrios Pendarakis, Ian Molloy
Date: 2018-12-07 00:00:00
Tasks: Data Poisoning

Distributed collaborative learning (DCL) paradigms enable building joint
machine learning models from distrusting multi-party participants. Data
confidentiality is guaranteed by retaining private training data on each
participant's local infrastructure. However, this approach to achieving data
confidentiality makes today's DCL designs fundamentally vulnerable to data
poisoning and backdoor attacks. It also limits DCL's model accountability,
which is key to backtracking the responsible "bad" training data
instances/contributors. In this paper, we introduce CALTRAIN, a Trusted
Execution Environment (TEE) based centralized multi-party collaborative
learning system that simultaneously achieves data confidentiality and model
accountability. CALTRAIN enforces isolated computation on centrally aggregated
training data to guarantee data confidentiality. To support building
accountable learning models, we securely maintain the links between training
instances and their corresponding contributors. Our evaluation shows that the
models generated from CALTRAIN can achieve the same prediction accuracy when
compared to the models trained in non-protected environments. We also
demonstrate that when malicious training participants tend to implant backdoors
during model training, CALTRAIN can accurately and precisely discover the
poisoned and mislabeled training data that lead to the runtime mispredictions.
A Hybrid Approach to Privacy-Preserving Federated Learning
Authors: Stacey Truex, Nathalie Baracaldo, Ali Anwar, Thomas Steinke, Heiko Ludwig, Rui Zhang, Yi Zhou
Date: 2018-12-07 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning, Privacy Preserving

Federated learning facilitates the collaborative training of models without the sharing of raw data. However, recent attacks demonstrate that simply maintaining data locality during training processes does not provide sufficient privacy guarantees. Rather, we need a federated learning system capable of preventing inference over both the messages exchanged during training and the final trained model while ensuring the resulting model also has acceptable predictive accuracy. Existing federated learning approaches either use secure multiparty computation (SMC) which is vulnerable to inference or differential privacy which can lead to low accuracy given a large number of parties with relatively small amounts of data each. In this paper, we present an alternative approach that utilizes both differential privacy and SMC to balance these trade-offs. Combining differential privacy with secure multiparty computation enables us to reduce the growth of noise injection as the number of parties increases without sacrificing privacy while maintaining a pre-defined rate of trust. Our system is therefore a scalable approach that protects against inference threats and produces models with high accuracy. Additionally, our system can be used to train a variety of machine learning models, which we validate with experimental results on 3 different machine learning algorithms. Our experiments demonstrate that our approach out-performs state of the art solutions.
Phenotype Inference with Semi-Supervised Mixed Membership Models
Authors: Victor Rodriguez, Adler Perotte
Date: 2018-12-07 00:00:00

Disease phenotyping algorithms process observational clinical data to
identify patients with specific diseases. Supervised phenotyping methods
require significant quantities of expert-labeled data, while unsupervised
methods may learn non-disease phenotypes. To address these limitations, we
propose the Semi-Supervised Mixed Membership Model (SS3M) -- a probabilistic
graphical model for learning disease phenotypes from clinical data with
relatively few labels. We show SS3M can learn interpretable, disease-specific
phenotypes which capture the clinical characteristics of the diseases specified
by the labels provided.