pwc-archive/papers-with-abstracts huggingface dataset:
Tex2Shape: Detailed Full Human Body Geometry From a Single Image
Authors: Thiemo Alldieck, Gerard Pons-Moll, Christian Theobalt, Marcus Magnor
Date: 2019-04-18 00:00:00
Proceeding: ICCV 2019 10
Tasks: Image-to-Image Translation, Translation

We present a simple yet effective method to infer detailed full human body shape from only a single photograph. Our model can infer full-body shape including face, hair, and clothing including wrinkles at interactive frame-rates. Results feature details even on parts that are occluded in the input image. Our main idea is to turn shape regression into an aligned image-to-image translation problem. The input to our method is a partial texture map of the visible region obtained from off-the-shelf methods. From a partial texture, we estimate detailed normal and vector displacement maps, which can be applied to a low-resolution smooth body model to add detail and clothing. Despite being trained purely with synthetic data, our model generalizes well to real-world photographs. Numerous results demonstrate the versatility and robustness of our method.
Real-Time Style Transfer With Strength Control
Authors: Victor Kitov
Date: 2019-04-18 00:00:00
Tasks: Style Transfer

Style transfer is a problem of rendering a content image in the style of
another style image. A natural and common practical task in applications of
style transfer is to adjust the strength of stylization. Algorithm of Gatys et
al. (2016) provides this ability by changing the weighting factors of content
and style losses but is computationally inefficient. Real-time style transfer
introduced by Johnson et al. (2016) enables fast stylization of any image by
passing it through a pre-trained transformer network. Although fast, this
architecture is not able to continuously adjust style strength. We propose an
extension to real-time style transfer that allows direct control of style
strength at inference, still requiring only a single transformer network. We
conduct qualitative and quantitative experiments that demonstrate that the
proposed method is capable of smooth stylization strength control and removes
certain stylization artifacts appearing in the original real-time style
transfer method. Comparisons with alternative real-time style transfer
algorithms, capable of adjusting stylization strength, show that our method
reproduces style with more details.
ConvLab: Multi-Domain End-to-End Dialog System Platform
Authors: Sungjin Lee, Qi Zhu, Ryuichi Takanobu, Xiang Li, Yaoqin Zhang, Zheng Zhang, Jinchao Li, Baolin Peng, Xiujun Li, Minlie Huang, Jianfeng Gao
Date: 2019-04-18 00:00:00
Proceeding: ACL 2019 7

We present ConvLab, an open-source multi-domain end-to-end dialog system
platform, that enables researchers to quickly set up experiments with reusable
components and compare a large set of different approaches, ranging from
conventional pipeline systems to end-to-end neural models, in common
environments. ConvLab offers a set of fully annotated datasets and associated
pre-trained reference models. As a showcase, we extend the MultiWOZ dataset
with user dialog act annotations to train all component models and demonstrate
how ConvLab makes it easy and effortless to conduct complicated experiments in
multi-domain end-to-end dialog settings.
DDLSTM: Dual-Domain LSTM for Cross-Dataset Action Recognition
Authors: Toby Perrett, Dima Damen
Date: 2019-04-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: Action Recognition, Temporal Action Localization

Domain alignment in convolutional networks aims to learn the degree of
layer-specific feature alignment beneficial to the joint learning of source and
target datasets. While increasingly popular in convolutional networks, there
have been no previous attempts to achieve domain alignment in recurrent
networks. Similar to spatial features, both source and target domains are
likely to exhibit temporal dependencies that can be jointly learnt and aligned.
  In this paper we introduce Dual-Domain LSTM (DDLSTM), an architecture that is
able to learn temporal dependencies from two domains concurrently. It performs
cross-contaminated batch normalisation on both input-to-hidden and
hidden-to-hidden weights, and learns the parameters for cross-contamination,
for both single-layer and multi-layer LSTM architectures. We evaluate DDLSTM on
frame-level action recognition using three datasets, taking a pair at a time,
and report an average increase in accuracy of 3.5%. The proposed DDLSTM
architecture outperforms standard, fine-tuned, and batch-normalised LSTMs.
Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data
Authors: Ke Gu, DaCheng Tao, Junfei Qiao, Weisi Lin
Date: 2019-04-18 00:00:00
Tasks: Image Enhancement, Image Quality Assessment, NR-IQA, object-detection, Object Detection

In this paper we investigate into the problem of image quality assessment
(IQA) and enhancement via machine learning. This issue has long attracted a
wide range of attention in computational intelligence and image processing
communities, since, for many practical applications, e.g. object detection and
recognition, raw images are usually needed to be appropriately enhanced to
raise the visual quality (e.g. visibility and contrast). In fact, proper
enhancement can noticeably improve the quality of input images, even better
than originally captured images which are generally thought to be of the best
quality. In this work, we present two most important contributions. The first
contribution is to develop a new no-reference (NR) IQA model. Given an image,
our quality measure first extracts 17 features through analysis of contrast,
sharpness, brightness and more, and then yields a measre of visual quality
using a regression module, which is learned with big-data training samples that
are much bigger than the size of relevant image datasets. Results of
experiments on nine datasets validate the superiority and efficiency of our
blind metric compared with typical state-of-the-art full-, reduced- and
no-reference IQA methods. The second contribution is that a robust image
enhancement framework is established based on quality optimization. For an
input image, by the guidance of the proposed NR-IQA measure, we conduct
histogram modification to successively rectify image brightness and contrast to
a proper level. Thorough tests demonstrate that our framework can well enhance
natural images, low-contrast images, low-light images and dehazed images. The
source code will be released at
https://sites.google.com/site/guke198701/publications.
Unsupervised Open Domain Recognition by Semantic Discrepancy Minimization
Authors: Junbao Zhuo, Shuhui Wang, Shuhao Cui, Qingming Huang
Date: 2019-04-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: Classification, General Classification

We address the unsupervised open domain recognition (UODR) problem, where
categories in labeled source domain S is only a subset of those in unlabeled
target domain T. The task is to correctly classify all samples in T including
known and unknown categories. UODR is challenging due to the domain
discrepancy, which becomes even harder to bridge when a large number of unknown
categories exist in T. Moreover, the classification rules propagated by graph
CNN (GCN) may be distracted by unknown categories and lack generalization
capability. To measure the domain discrepancy for asymmetric label space
between S and T, we propose Semantic-Guided Matching Discrepancy (SGMD), which
first employs instance matching between S and T, and then the discrepancy is
measured by a weighted feature distance between matched instances. We further
design a limited balance constraint to achieve a more balanced classification
output on known and unknown categories. We develop Unsupervised Open Domain
Transfer Network (UODTN), which learns both the backbone classification network
and GCN jointly by reducing the SGMD, enforcing the limited balance constraint
and minimizing the classification loss on S. UODTN better preserves the
semantic structure and enforces the consistency between the learned domain
invariant visual features and the semantic embeddings. Experimental results
show superiority of our method on recognizing images of both known and unknown
categories.
Discriminative Online Learning for Fast Video Object Segmentation
Authors: Andreas Robinson, Felix Järemo Lawin, Martin Danelljan, Fahad Shahbaz Khan, Michael Felsberg
Date: 2019-04-18 00:00:00
Tasks: Object, One-shot visual object segmentation, Segmentation, Semantic Segmentation, Video Object Segmentation, Video Semantic Segmentation

We address the highly challenging problem of video object segmentation. Given
only the initial mask, the task is to segment the target in the subsequent
frames. In order to effectively handle appearance changes and similar
background objects, a robust representation of the target is required. Previous
approaches either rely on fine-tuning a segmentation network on the first
frame, or employ generative appearance models. Although partially successful,
these methods often suffer from impractically low frame rates or unsatisfactory
robustness.
  We propose a novel approach, based on a dedicated target appearance model
that is exclusively learned online to discriminate between the target and
background image regions. Importantly, we design a specialized loss and
customized optimization techniques to enable highly efficient online training.
Our light-weight target model is integrated into a carefully designed
segmentation network, trained offline to enhance the predictions generated by
the target model. Extensive experiments are performed on three datasets. Our
approach achieves an overall score of over 70 on YouTube-VOS, while operating
at 25 frames per second.
Ontology-based Design of Experiments on Big Data Solutions
Authors: Maximilian Zocholl, Elena Camossi, Anne-Laure Jousselme, Cyril Ray
Date: 2019-04-18 00:00:00

Big data solutions are designed to cope with data of huge Volume and wide
Variety, that need to be ingested at high Velocity and have potential Veracity
issues, challenging characteristics that are usually referred to as the "4Vs of
Big Data". In order to evaluate possibly complex big data solutions, stress
tests require to assess a large number of combinations of sub-components
jointly with the possible big data variations. A formalization of the Design of
Experiments (DoE) on big data solutions is aimed at ensuring the
reproducibility of the experiments, facilitating their partitioning in
sub-experiments and guaranteeing the consistency of their outcomes in a global
assessment. In this paper, an ontology-based approach is proposed to support
the evaluation of a big data system in two ways. Firstly, the approach
formalizes a decomposition and recombination of the big data solution, allowing
for the aggregation of component evaluation results at inter-component level.
Secondly, existing work on DoE is translated into an ontology for supporting
the selection of experiments. The proposed ontology-based approach offers the
possibility to combine knowledge from the evaluation domain and the application
domain. It exploits domain and inter-domain specific restrictions on the factor
combinations in order to reduce the number of experiments. Contrary to existing
approaches, the proposed use of ontologies is not limited to the assertional
description and exploitation of past experiments but offers richer
terminological descriptions for the development of a DoE from scratch. As an
application example, a maritime big data solution to the problem of detecting
and predicting vessel suspicious behaviour through mobility analysis is
selected. The article is concluded with a sketch of future works.
Query-Adaptive Hash Code Ranking for Large-Scale Multi-View Visual Search
Authors: Xianglong Liu, Lei Huang, Cheng Deng, Bo Lang, DaCheng Tao
Date: 2019-04-18 00:00:00
Tasks: Image Retrieval, Quantization, Table Search

Hash based nearest neighbor search has become attractive in many
applications. However, the quantization in hashing usually degenerates the
discriminative power when using Hamming distance ranking. Besides, for
large-scale visual search, existing hashing methods cannot directly support the
efficient search over the data with multiple sources, and while the literature
has shown that adaptively incorporating complementary information from diverse
sources or views can significantly boost the search performance. To address the
problems, this paper proposes a novel and generic approach to building multiple
hash tables with multiple views and generating fine-grained ranking results at
bitwise and tablewise levels. For each hash table, a query-adaptive bitwise
weighting is introduced to alleviate the quantization loss by simultaneously
exploiting the quality of hash functions and their complement for nearest
neighbor search. From the tablewise aspect, multiple hash tables are built for
different data views as a joint index, over which a query-specific rank fusion
is proposed to rerank all results from the bitwise ranking by diffusing in a
graph. Comprehensive experiments on image search over three well-known
benchmarks show that the proposed method achieves up to 17.11% and 20.28%
performance gains on single and multiple table search over state-of-the-art
methods.
Dimensionality Reduction of Complex Metastable Systems via Kernel Embeddings of Transition Manifolds
Authors: Andreas Bittracher, Stefan Klus, Boumediene Hamzi, Péter Koltai, Christof Schütte
Date: 2019-04-18 00:00:00
Tasks: Dimensionality Reduction

We present a novel kernel-based machine learning algorithm for identifying the low-dimensional geometry of the effective dynamics of high-dimensional multiscale stochastic systems. Recently, the authors developed a mathematical framework for the computation of optimal reaction coordinates of such systems that is based on learning a parametrization of a low-dimensional transition manifold in a certain function space. In this article, we enhance this approach by embedding and learning this transition manifold in a reproducing kernel Hilbert space, exploiting the favorable properties of kernel embeddings. Under mild assumptions on the kernel, the manifold structure is shown to be preserved under the embedding, and distortion bounds can be derived. This leads to a more robust and more efficient algorithm compared to previous parametrization approaches.
Improving Interactive Reinforcement Agent Planning with Human Demonstration
Authors: Guangliang Li, Randy Gomez, Keisuke Nakamura, Jinying Lin, Qilei Zhang, Bo He
Date: 2019-04-18 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

TAMER has proven to be a powerful interactive reinforcement learning method
for allowing ordinary people to teach and personalize autonomous agents'
behavior by providing evaluative feedback. However, a TAMER agent planning with
UCT---a Monte Carlo Tree Search strategy, can only update states along its path
and might induce high learning cost especially for a physical robot. In this
paper, we propose to drive the agent's exploration along the optimal path and
reduce the learning cost by initializing the agent's reward function via
inverse reinforcement learning from demonstration. We test our proposed method
in the RL benchmark domain---Grid World---with different discounts on human
reward. Our results show that learning from demonstration can allow a TAMER
agent to learn a roughly optimal policy up to the deepest search and encourage
the agent to explore along the optimal path. In addition, we find that learning
from demonstration can improve the learning efficiency by reducing total
feedback, the number of incorrect actions and increasing the ratio of correct
actions to obtain an optimal policy, allowing a TAMER agent to converge faster.
Disentangled Representation Learning with Information Maximizing Autoencoder
Authors: Kazi Nazmul Haque, Siddique Latif, Rajib Rana
Date: 2019-04-18 00:00:00
Tasks: Representation Learning

Learning disentangled representation from any unlabelled data is a
non-trivial problem. In this paper we propose Information Maximising
Autoencoder (InfoAE) where the encoder learns powerful disentangled
representation through maximizing the mutual information between the
representation and given information in an unsupervised fashion. We have
evaluated our model on MNIST dataset and achieved 98.9 ($\pm .1$) $\%$ test
accuracy while using complete unsupervised training.
Client/Server Based Online Environment for Manual Segmentation of Medical Images
Authors: Daniel Wild, Maximilian Weber, Jan Egger
Date: 2019-04-18 00:00:00
Tasks: Segmentation

Segmentation is a key step in analyzing and processing medical images. Due to
the low fault tolerance in medical imaging, manual segmentation remains the de
facto standard in this domain. Besides, efforts to automate the segmentation
process often rely on large amounts of manually labeled data. While existing
software supporting manual segmentation is rich in features and delivers
accurate results, the necessary time to set it up and get comfortable using it
can pose a hurdle for the collection of large datasets. This work introduces a
client/server based online environment, referred to as Studierfenster
(studierfenster.at), that can be used to perform manual segmentations directly
in a web browser. The aim of providing this functionality in the form of a web
application is to ease the collection of ground truth segmentation datasets.
Providing a tool that is quickly accessible and usable on a broad range of
devices, offers the potential to accelerate this process. The manual
segmentation workflow of Studierfenster consists of dragging and dropping the
input file into the browser window and slice-by-slice outlining the object
under consideration. The final segmentation can then be exported as a file
storing its contours and as a binary segmentation mask. In order to evaluate
the usability of Studierfenster, a user study was performed. The user study
resulted in a mean of 6.3 out of 7.0 possible points given by users, when asked
about their overall impression of the tool. The evaluation also provides
insights into the results achievable with the tool in practice, by presenting
two ground truth segmentations performed by physicians.
Learning to Collocate Neural Modules for Image Captioning
Authors: Xu Yang, Hanwang Zhang, Jianfei Cai
Date: 2019-04-18 00:00:00
Proceeding: ICCV 2019 10
Tasks: Decoder, Image Captioning, Sentence, Visual Question Answering (VQA), Visual Reasoning

We do not speak word by word from scratch; our brain quickly structures a
pattern like \textsc{sth do sth at someplace} and then fill in the detailed
descriptions. To render existing encoder-decoder image captioners such
human-like reasoning, we propose a novel framework: learning to Collocate
Neural Modules (CNM), to generate the `inner pattern' connecting visual encoder
and language decoder. Unlike the widely-used neural module networks in visual
Q\&A, where the language (ie, question) is fully observable, CNM for captioning
is more challenging as the language is being generated and thus is partially
observable. To this end, we make the following technical contributions for CNM
training: 1) compact module design --- one for function words and three for
visual content words (eg, noun, adjective, and verb), 2) soft module fusion and
multi-step module execution, robustifying the visual reasoning in partial
observation, 3) a linguistic loss for module controller being faithful to
part-of-speech collocations (eg, adjective is before noun). Extensive
experiments on the challenging MS-COCO image captioning benchmark validate the
effectiveness of our CNM image captioner. In particular, CNM achieves a new
state-of-the-art 127.9 CIDEr-D on Karpathy split and a single-model 126.0 c40
on the official server. CNM is also robust to few training samples, eg, by
training only one sentence per image, CNM can halve the performance loss
compared to a strong baseline.
Progressive Attention Memory Network for Movie Story Question Answering
Authors: Junyeong Kim, Minuk Ma, Kyung-Su Kim, Sungjin Kim, Chang D. Yoo
Date: 2019-04-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: Question Answering, Video Story QA, Visual Question Answering (VQA)

This paper proposes the progressive attention memory network (PAMN) for movie
story question answering (QA). Movie story QA is challenging compared to VQA in
two aspects: (1) pinpointing the temporal parts relevant to answer the question
is difficult as the movies are typically longer than an hour, (2) it has both
video and subtitle where different questions require different modality to
infer the answer. To overcome these challenges, PAMN involves three main
features: (1) progressive attention mechanism that utilizes cues from both
question and answer to progressively prune out irrelevant temporal parts in
memory, (2) dynamic modality fusion that adaptively determines the contribution
of each modality for answering the current question, and (3) belief correction
answering scheme that successively corrects the prediction score on each
candidate answer. Experiments on publicly available benchmark datasets, MovieQA
and TVQA, demonstrate that each feature contributes to our movie story QA
architecture, PAMN, and improves performance to achieve the state-of-the-art
result. Qualitative analysis by visualizing the inference mechanism of PAMN is
also provided.
Deep Optics for Monocular Depth Estimation and 3D Object Detection
Authors: Julie Chang, Gordon Wetzstein
Date: 2019-04-18 00:00:00
Proceeding: ICCV 2019 10
Tasks: 3D Object Detection, Depth Estimation, Monocular Depth Estimation, Object, object-detection, Object Detection, Scene Understanding

Depth estimation and 3D object detection are critical for scene understanding
but remain challenging to perform with a single image due to the loss of 3D
information during image capture. Recent models using deep neural networks have
improved monocular depth estimation performance, but there is still difficulty
in predicting absolute depth and generalizing outside a standard dataset. Here
we introduce the paradigm of deep optics, i.e. end-to-end design of optics and
image processing, to the monocular depth estimation problem, using coded
defocus blur as an additional depth cue to be decoded by a neural network. We
evaluate several optical coding strategies along with an end-to-end
optimization scheme for depth estimation on three datasets, including NYU Depth
v2 and KITTI. We find an optimized freeform lens design yields the best
results, but chromatic aberration from a singlet lens offers significantly
improved performance as well. We build a physical prototype and validate that
chromatic aberrations improve depth estimation on real-world results. In
addition, we train object detection networks on the KITTI dataset and show that
the lens optimized for depth estimation also results in improved 3D object
detection performance.
Reducing Noise in GAN Training with Variance Reduced Extragradient
Authors: Tatjana Chavdarova, Gauthier Gidel, François Fleuret, Simon Lacoste-Julien
Date: 2019-04-18 00:00:00
Proceeding: NeurIPS 2019 12

We study the effect of the stochastic gradient noise on the training of generative adversarial networks (GANs) and show that it can prevent the convergence of standard game optimization methods, while the batch version converges. We address this issue with a novel stochastic variance-reduced extragradient (SVRE) optimization algorithm, which for a large class of games improves upon the previous convergence rates proposed in the literature. We observe empirically that SVRE performs similarly to a batch method on MNIST while being computationally cheaper, and that SVRE yields more stable GAN training on standard datasets.