pwc-archive/papers-with-abstracts huggingface dataset:
Med3D: Transfer Learning for 3D Medical Image Analysis
Authors: Sihong Chen, Kai Ma, Yefeng Zheng
Date: 2019-04-01 00:00:00
Tasks: 3D Medical Imaging Segmentation, Liver Segmentation, Medical Image Analysis, Segmentation, Transfer Learning

The performance on deep learning is significantly affected by volume of training data. Models pre-trained from massive dataset such as ImageNet become a powerful weapon for speeding up training convergence and improving accuracy. Similarly, models based on large dataset are important for the development of deep learning in 3D medical images. However, it is extremely challenging to build a sufficiently large dataset due to difficulty of data acquisition and annotation in 3D medical imaging. We aggregate the dataset from several medical challenges to build 3DSeg-8 dataset with diverse modalities, target organs, and pathologies. To extract general medical three-dimension (3D) features, we design a heterogeneous 3D network called Med3D to co-train multi-domain 3DSeg-8 so as to make a series of pre-trained models. We transfer Med3D pre-trained models to lung segmentation in LIDC dataset, pulmonary nodule classification in LIDC dataset and liver segmentation on LiTS challenge. Experiments show that the Med3D can accelerate the training convergence speed of target 3D medical tasks 2 times compared with model pre-trained on Kinetics dataset, and 10 times compared with training from scratch as well as improve accuracy ranging from 3% to 20%. Transferring our Med3D model on state-the-of-art DenseASPP segmentation network, in case of single model, we achieve 94.6\% Dice coefficient which approaches the result of top-ranged algorithms on the LiTS challenge.
TAN: Temporal Affine Network for Real-Time Left Ventricle Anatomical Structure Analysis Based on 2D Ultrasound Videos
Authors: Sihong Chen, Kai Ma, Yefeng Zheng
Date: 2019-04-01 00:00:00
Tasks: LV Segmentation, Optical Flow Estimation, Segmentation

With superiorities on low cost, portability, and free of radiation,
echocardiogram is a widely used imaging modality for left ventricle (LV)
function quantification. However, automatic LV segmentation and motion tracking
is still a challenging task. In addition to fuzzy border definition, low
contrast, and abounding artifacts on typical ultrasound images, the shape and
size of the LV change significantly in a cardiac cycle. In this work, we
propose a temporal affine network (TAN) to perform image analysis in a warped
image space, where the shape and size variations due to the cardiac motion as
well as other artifacts are largely compensated. Furthermore, we perform three
frequent echocardiogram interpretation tasks simultaneously: standard cardiac
plane recognition, LV landmark detection, and LV segmentation. Instead of using
three networks with one dedicating to each task, we use a multi-task network to
perform three tasks simultaneously. Since three tasks share the same encoder,
the compact network improves the segmentation accuracy with more supervision.
The network is further finetuned with optical flow adjusted annotations to
enhance motion coherence in the segmentation result. Experiments on 1,714 2D
echocardiographic sequences demonstrate that the proposed method achieves
state-of-the-art segmentation accuracy with real-time efficiency.
CFSNet: Toward a Controllable Feature Space for Image Restoration
Authors: Wei Wang, Ruiming Guo, Yapeng Tian, Wenming Yang
Date: 2019-04-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: Image Restoration, Image Super-Resolution, SSIM, Super-Resolution

Deep learning methods have witnessed the great progress in image restoration with specific metrics (e.g., PSNR, SSIM). However, the perceptual quality of the restored image is relatively subjective, and it is necessary for users to control the reconstruction result according to personal preferences or image characteristics, which cannot be done using existing deterministic networks. This motivates us to exquisitely design a unified interactive framework for general image restoration tasks. Under this framework, users can control continuous transition of different objectives, e.g., the perception-distortion trade-off of image super-resolution, the trade-off between noise reduction and detail preservation. We achieve this goal by controlling the latent features of the designed network. To be specific, our proposed framework, named Controllable Feature Space Network (CFSNet), is entangled by two branches based on different objectives. Our framework can adaptively learn the coupling coefficients of different layers and channels, which provides finer control of the restored image quality. Experiments on several typical image restoration tasks fully validate the effective benefits of the proposed method. Code is available at https://github.com/qibao77/CFSNet.
Harvesting Visual Objects from Internet Images via Deep Learning Based Objectness Assessment
Authors: Kan Wu, Guanbin Li, Haofeng Li, Jianjun Zhang, Yizhou Yu
Date: 2019-04-01 00:00:00
Tasks: Image Generation, Object, Re-Ranking

The collection of internet images has been growing in an astonishing speed.
It is undoubted that these images contain rich visual information that can be
useful in many applications, such as visual media creation and data-driven
image synthesis. In this paper, we focus on the methodologies for building a
visual object database from a collection of internet images. Such database is
built to contain a large number of high-quality visual objects that can help
with various data-driven image applications. Our method is based on dense
proposal generation and objectness-based re-ranking. A novel deep convolutional
neural network is designed for the inference of proposal objectness, the
probability of a proposal containing optimally-located foreground object. In
our work, the objectness is quantitatively measured in regard of completeness
and fullness, reflecting two complementary features of an optimal proposal: a
complete foreground and relatively small background. Our experiments indicate
that object proposals re-ranked according to the output of our network
generally achieve higher performance than those produced by other
state-of-the-art methods. As a concrete example, a database of over 1.2 million
visual objects has been built using the proposed method, and has been
successfully used in various data-driven image applications.
Deep Learning for Large-Scale Traffic-Sign Detection and Recognition
Authors: Domen Tabernik, Danijel Skočaj
Date: 2019-04-01 00:00:00
Tasks: Deep Learning, Management, Open-Ended Question Answering, Traffic Sign Detection, Traffic Sign Recognition

Automatic detection and recognition of traffic signs plays a crucial role in
management of the traffic-sign inventory. It provides accurate and timely way
to manage traffic-sign inventory with a minimal human effort. In the computer
vision community the recognition and detection of traffic signs is a
well-researched problem. A vast majority of existing approaches perform well on
traffic signs needed for advanced drivers-assistance and autonomous systems.
However, this represents a relatively small number of all traffic signs (around
50 categories out of several hundred) and performance on the remaining set of
traffic signs, which are required to eliminate the manual labor in traffic-sign
inventory management, remains an open question. In this paper, we address the
issue of detecting and recognizing a large number of traffic-sign categories
suitable for automating traffic-sign inventory management. We adopt a
convolutional neural network (CNN) approach, the Mask R-CNN, to address the
full pipeline of detection and recognition with automatic end-to-end learning.
We propose several improvements that are evaluated on the detection of traffic
signs and result in an improved overall performance. This approach is applied
to detection of 200 traffic-sign categories represented in our novel dataset.
Results are reported on highly challenging traffic-sign categories that have
not yet been considered in previous works. We provide comprehensive analysis of
the deep learning method for the detection of traffic signs with large
intra-category appearance variation and show below 3% error rates with the
proposed approach, which is sufficient for deployment in practical applications
of traffic-sign inventory management.
Learning Content-Weighted Deep Image Compression
Authors: Mu Li, WangMeng Zuo, Shuhang Gu, Jane You, David Zhang
Date: 2019-04-01 00:00:00
Tasks: Decoder, Image Compression

Learning-based lossy image compression usually involves the joint
optimization of rate-distortion performance. Most existing methods adopt
spatially invariant bit length allocation and incorporate discrete entropy
approximation to constrain compression rate. Nonetheless, the information
content is spatially variant, where the regions with complex and salient
structures generally are more essential to image compression. Taking the
spatial variation of image content into account, this paper presents a
content-weighted encoder-decoder model, which involves an importance map subnet
to produce the importance mask for locally adaptive bit rate allocation.
Consequently, the summation of importance mask can thus be utilized as an
alternative of entropy estimation for compression rate control. Furthermore,
the quantized representations of the learned code and importance map are still
spatially dependent, which can be losslessly compressed using arithmetic
coding. To compress the codes effectively and efficiently, we propose a trimmed
convolutional network to predict the conditional probability of quantized
codes. Experiments show that the proposed method can produce visually much
better results, and performs favorably in comparison with deep and traditional
lossy image compression approaches.
Deep Built-Structure Counting in Satellite Imagery Using Attention Based Re-Weighting
Authors: Anza Shakeel, Waqas Sultani, Mohsen Ali
Date: 2019-04-01 00:00:00

In this paper, we attempt to address the challenging problem of counting
built-structures in the satellite imagery. Building density is a more accurate
estimate of the population density, urban area expansion and its impact on the
environment, than the built-up area segmentation. However, building shape
variances, overlapping boundaries, and variant densities make this a complex
task. To tackle this difficult problem, we propose a deep learning based
regression technique for counting built-structures in satellite imagery. Our
proposed framework intelligently combines features from different regions of
satellite image using attention based re-weighting techniques. Multiple
parallel convolutional networks are designed to capture information at
different granulates. These features are combined into the FusionNet which is
trained to weigh features from different granularity differently, allowing us
to predict a precise building count. To train and evaluate the proposed method,
we put forward a new large-scale and challenging built-structure-count dataset.
Our dataset is constructed by collecting satellite imagery from diverse
geographical areas (planes, urban centers, deserts, etc.,) across the globe
(Asia, Europe, North America, and Africa) and captures the wide density of
built structures. Detailed experimental results and analysis validate the
proposed technique. FusionNet has Mean Absolute Error of 3.65 and R-squared
measure of 88% over the testing data. Finally, we perform the test on the 274:3
? 103 m2 of the unseen region, with the error of 19 buildings off the 656
buildings in that area.
Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge
Authors: Hugo J. Kuijf, J. Matthijs Biesbroek, Jeroen de Bresser, Rutger Heinen, Simon Andermatt, Mariana Bento, Matt Berseth, Mikhail Belyaev, M. Jorge Cardoso, Adrià Casamitjana, D. Louis Collins, Mahsa Dadar, Achilleas Georgiou, Mohsen Ghafoorian, Dakai Jin, April Khademi, Jesse Knight, Hongwei Li, Xavier Lladó, Miguel Luna, Qaiser Mahmood, Richard McKinley, Alireza Mehrtash, Sébastien Ourselin, Bo-yong Park, HyunJin Park, Sang Hyun Park, Simon Pezold, Elodie Puybareau, Leticia Rittner, Carole H. Sudre, Sergi Valverde, Verónica Vilaplana, Roland Wiest, Yongchao Xu, Ziyue Xu, Guodong Zeng, Jian-Guo Zhang, Guoyan Zheng, Christopher Chen, Wiesje van der Flier, Frederik Barkhof, Max A. Viergever, Geert Jan Biessels
Date: 2019-04-01 00:00:00
Tasks: Segmentation

Quantification of cerebral white matter hyperintensities (WMH) of presumed
vascular origin is of key importance in many neurological research studies.
Currently, measurements are often still obtained from manual segmentations on
brain MR images, which is a laborious procedure. Automatic WMH segmentation
methods exist, but a standardized comparison of the performance of such methods
is lacking. We organized a scientific challenge, in which developers could
evaluate their method on a standardized multi-center/-scanner image dataset,
giving an objective comparison: the WMH Segmentation Challenge
(https://wmh.isi.uu.nl/).
  Sixty T1+FLAIR images from three MR scanners were released with manual WMH
segmentations for training. A test set of 110 images from five MR scanners was
used for evaluation. Segmentation methods had to be containerized and submitted
to the challenge organizers. Five evaluation metrics were used to rank the
methods: (1) Dice similarity coefficient, (2) modified Hausdorff distance (95th
percentile), (3) absolute log-transformed volume difference, (4) sensitivity
for detecting individual lesions, and (5) F1-score for individual lesions.
Additionally, methods were ranked on their inter-scanner robustness.
  Twenty participants submitted their method for evaluation. This paper
provides a detailed analysis of the results. In brief, there is a cluster of
four methods that rank significantly better than the other methods, with one
clear winner. The inter-scanner robustness ranking shows that not all methods
generalize to unseen scanners.
  The challenge remains open for future submissions and provides a public
platform for method evaluation.
Dance with Flow: Two-in-One Stream Action Detection
Authors: Jiaojiao Zhao, Cees G. M. Snoek
Date: 2019-04-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Action Detection, Optical Flow Estimation, Vocal Bursts Valence Prediction

The goal of this paper is to detect the spatio-temporal extent of an action. The two-stream detection network based on RGB and flow provides state-of-the-art accuracy at the expense of a large model-size and heavy computation. We propose to embed RGB and optical-flow into a single two-in-one stream network with new layers. A motion condition layer extracts motion information from flow images, which is leveraged by the motion modulation layer to generate transformation parameters for modulating the low-level RGB features. The method is easily embedded in existing appearance- or two-stream action detection networks, and trained end-to-end. Experiments demonstrate that leveraging the motion condition to modulate RGB features improves detection accuracy. With only half the computation and parameters of the state-of-the-art two-stream methods, our two-in-one stream still achieves impressive results on UCF101-24, UCFSports and J-HMDB.
GAN You Do the GAN GAN?
Authors: Joseph Suarez
Date: 2019-04-01 00:00:00
Tasks: Image Generation

Generative Adversarial Networks (GANs) have become a dominant class of
generative models. In recent years, GAN variants have yielded especially
impressive results in the synthesis of a variety of forms of data. Examples
include compelling natural and artistic images, textures, musical sequences,
and 3D object files. However, one obvious synthesis candidate is missing. In
this work, we answer one of deep learning's most pressing questions: GAN you do
the GAN GAN? That is, is it possible to train a GAN to model a distribution of
GANs? We release the full source code for this project under the MIT license.
Unsupervised Multi-modal Hashing for Cross-modal retrieval
Authors: Jun Yu, Xiao-Jun Wu
Date: 2019-03-26 00:00:00
Tasks: Content-Based Image Retrieval, Cross-Modal Retrieval, Graph Embedding, Image Retrieval, Retrieval

With the advantage of low storage cost and high efficiency, hashing learning has received much attention in the domain of Big Data. In this paper, we propose a novel unsupervised hashing learning method to cope with this open problem to directly preserve the manifold structure by hashing. To address this problem, both the semantic correlation in textual space and the locally geometric structure in the visual space are explored simultaneously in our framework. Besides, the `2;1-norm constraint is imposed on the projection matrices to learn the discriminative hash function for each modality. Extensive experiments are performed to evaluate the proposed method on the three publicly available datasets and the experimental results show that our method can achieve superior performance over the state-of-the-art methods.
Semantic Nearest Neighbor Fields Monocular Edge Visual-Odometry
Authors: Xiaolong Wu, Assia Benbihi, Antoine Richard, Cedric Pradalier
Date: 2019-04-01 00:00:00
Tasks: Edge Detection, Monocular Visual Odometry, Motion Estimation, Visual Odometry

Recent advances in deep learning for edge detection and segmentation opens up
a new path for semantic-edge-based ego-motion estimation. In this work, we
propose a robust monocular visual odometry (VO) framework using category-aware
semantic edges. It can reconstruct large-scale semantic maps in challenging
outdoor environments. The core of our approach is a semantic nearest neighbor
field that facilitates a robust data association of edges across frames using
semantics. This significantly enlarges the convergence radius during tracking
phases. The proposed edge registration method can be easily integrated into
direct VO frameworks to estimate photometrically, geometrically, and
semantically consistent camera motions. Different types of edges are evaluated
and extensive experiments demonstrate that our proposed system outperforms
state-of-art indirect, direct, and semantic monocular VO systems.
Through-Wall Pose Imaging in Real-Time with a Many-to-Many Encoder/Decoder Paradigm
Authors: Kevin Meng, Yu Meng
Date: 2019-03-15 00:00:00
Tasks: Decoder, Region Proposal

Overcoming the visual barrier and developing "see-through vision" has been one of mankind's long-standing dreams. Unlike visible light, Radio Frequency (RF) signals penetrate opaque obstructions and reflect highly off humans. This paper establishes a deep-learning model that can be trained to reconstruct continuous video of a 15-point human skeleton even through visual occlusion. The training process adopts a student/teacher learning procedure inspired by the Feynman learning technique, in which video frames and RF data are first collected simultaneously using a co-located setup containing an optical camera and an RF antenna array transceiver. Next, the video frames are processed with a computer-vision-based gait analysis "teacher" module to generate ground-truth human skeletons for each frame. Then, the same type of skeleton is predicted from corresponding RF data using a "student" deep-learning model consisting of a Residual Convolutional Neural Network (CNN), Region Proposal Network (RPN), and Recurrent Neural Network with Long-Short Term Memory (LSTM) that 1) extracts spatial features from RF images, 2) detects all people present in a scene, and 3) aggregates information over many time-steps, respectively. The model is shown to both accurately and completely predict the pose of humans behind visual obstruction solely using RF signals. Primary academic contributions include the novel many-to-many imaging methodology, unique integration of RPN and LSTM networks, and original training pipeline.
Projectron -- A Shallow and Interpretable Network for Classifying Medical Images
Authors: Aditya Sriram, Shivam Kalra, H. R. Tizhoosh
Date: 2019-03-15 00:00:00

This paper introduces the `Projectron' as a new neural network architecture
that uses Radon projections to both classify and represent medical images. The
motivation is to build shallow networks which are more interpretable in the
medical imaging domain. Radon transform is an established technique that can
reconstruct images from parallel projections. The Projectron first applies
global Radon transform to each image using equidistant angles and then feeds
these transformations for encoding to a single layer of neurons followed by a
layer of suitable kernels to facilitate a linear separation of projections.
Finally, the Projectron provides the output of the encoding as an input to two
more layers for final classification. We validate the Projectron on five
publicly available datasets, a general dataset (namely MNIST) and four medical
datasets (namely Emphysema, IDC, IRMA, and Pneumonia). The results are
encouraging as we compared the Projectron's performance against MLPs with raw
images and Radon projections as inputs, respectively. Experiments clearly
demonstrate the potential of the proposed Projectron for
representing/classifying medical images.
Fashion Outfit Generation for E-commerce
Authors: Elaine M. Bettaney, Stephen R. Hardwick, Odysseas Zisimopoulos, Benjamin Paul Chamberlain
Date: 2019-03-18 00:00:00

Combining items of clothing into an outfit is a major task in fashion retail.
Recommending sets of items that are compatible with a particular seed item is
useful for providing users with guidance and inspiration, but is currently a
manual process that requires expert stylists and is therefore not scalable or
easy to personalise. We use a multilayer neural network fed by visual and
textual features to learn embeddings of items in a latent style space such that
compatible items of different types are embedded close to one another. We train
our model using the ASOS outfits dataset, which consists of a large number of
outfits created by professional stylists and which we release to the research
community. Our model shows strong performance in an offline outfit
compatibility prediction task. We use our model to generate outfits and for the
first time in this field perform an AB test, comparing our generated outfits to
those produced by a baseline model which matches appropriate product types but
uses no information on style. Users approved of outfits generated by our model
21% and 34% more frequently than those generated by the baseline model for
womenswear and menswear respectively.