pwc-archive/papers-with-abstracts huggingface dataset:
Visual Entailment Task for Visually-Grounded Language Learning
Authors: Ning Xie, Farley Lai, Derek Doran, Asim Kadav
Date: 2018-11-26 00:00:00
Tasks: Grounded language learning, Natural Language Inference, Question Answering, Sentence, Visual Entailment, Visual Question Answering, Visual Question Answering (VQA)

We introduce a new inference task - Visual Entailment (VE) - which differs
from traditional Textual Entailment (TE) tasks whereby a premise is defined by
an image, rather than a natural language sentence as in TE tasks. A novel
dataset SNLI-VE (publicly available at https://github.com/necla-ml/SNLI-VE) is
proposed for VE tasks based on the Stanford Natural Language Inference corpus
and Flickr30k. We introduce a differentiable architecture called the
Explainable Visual Entailment model (EVE) to tackle the VE problem. EVE and
several other state-of-the-art visual question answering (VQA) based models are
evaluated on the SNLI-VE dataset, facilitating grounded language understanding
and providing insights on how modern VQA based models perform.
HOGWILD!-Gibbs can be PanAccurate
Authors: Constantinos Daskalakis, Nishanth Dikkala, Siddhartha Jayanti
Date: 2018-11-26 00:00:00
Proceeding: NeurIPS 2018 12

Asynchronous Gibbs sampling has been recently shown to be fast-mixing and an
accurate method for estimating probabilities of events on a small number of
variables of a graphical model satisfying Dobrushin's
condition~\cite{DeSaOR16}. We investigate whether it can be used to accurately
estimate expectations of functions of {\em all the variables} of the model.
Under the same condition, we show that the synchronous (sequential) and
asynchronous Gibbs samplers can be coupled so that the expected Hamming
distance between their (multivariate) samples remains bounded by $O(\tau \log
n),$ where $n$ is the number of variables in the graphical model, and $\tau$ is
a measure of the asynchronicity. A similar bound holds for any constant power
of the Hamming distance. Hence, the expectation of any function that is
Lipschitz with respect to a power of the Hamming distance, can be estimated
with a bias that grows logarithmically in $n$. Going beyond Lipschitz
functions, we consider the bias arising from asynchronicity in estimating the
expectation of polynomial functions of all variables in the model. Using recent
concentration of measure results, we show that the bias introduced by the
asynchronicity is of smaller order than the standard deviation of the function
value already present in the true model. We perform experiments on a
multi-processor machine to empirically illustrate our theoretical findings.
Stacked Spatio-Temporal Graph Convolutional Networks for Action Segmentation
Authors: Pallabi Ghosh, Yi Yao, Larry S. Davis, Ajay Divakaran
Date: 2018-11-26 00:00:00
Tasks: Action Recognition, Action Segmentation, Decoder, Skeleton Based Action Recognition, Temporal Action Localization

We propose novel Stacked Spatio-Temporal Graph Convolutional Networks (Stacked-STGCN) for action segmentation, i.e., predicting and localizing a sequence of actions over long videos. We extend the Spatio-Temporal Graph Convolutional Network (STGCN) originally proposed for skeleton-based action recognition to enable nodes with different characteristics (e.g., scene, actor, object, action, etc.), feature descriptors with varied lengths, and arbitrary temporal edge connections to account for large graph deformation commonly associated with complex activities. We further introduce the stacked hourglass architecture to STGCN to leverage the advantages of an encoder-decoder design for improved generalization performance and localization accuracy. We explore various descriptors such as frame-level VGG, segment-level I3D, RCNN-based object, etc. as node descriptors to enable action segmentation based on joint inference over comprehensive contextual information. We show results on CAD120 (which provides pre-computed node features and edge weights for fair performance comparison across algorithms) as well as a more complex real-world activity dataset, Charades. Our Stacked-STGCN in general achieves 4.0% performance improvement over the best reported results in F1 score on CAD120 and 1.3% in mAP on Charades using VGG features.
Convolutional Neural Networks Deceived by Visual Illusions
Authors: Alexander Gomez-Villa, Adrián Martín, Javier Vazquez-Corral, Marcelo Bertalmío
Date: 2018-11-26 00:00:00
Tasks: Color Constancy, Deblurring, Denoising, Image Deblurring, Image Denoising

Visual illusions teach us that what we see is not always what it is
represented in the physical world. Its special nature make them a fascinating
tool to test and validate any new vision model proposed. In general, current
vision models are based on the concatenation of linear convolutions and
non-linear operations. In this paper we get inspiration from the similarity of
this structure with the operations present in Convolutional Neural Networks
(CNNs). This motivated us to study if CNNs trained for low-level visual tasks
are deceived by visual illusions. In particular, we show that CNNs trained for
image denoising, image deblurring, and computational color constancy are able
to replicate the human response to visual illusions, and that the extent of
this replication varies with respect to variation in architecture and spatial
pattern size. We believe that this CNNs behaviour appears as a by-product of
the training for the low level vision tasks of denoising, color constancy or
deblurring. Our work opens a new bridge between human perception and CNNs: in
order to obtain CNNs that better replicate human behaviour, we may need to
start aiming for them to better replicate visual illusions.
Low-Dose CT via Deep CNN with Skip Connection and Network in Network
Authors: Chenyu You, Linfeng Yang, Yi Zhang, Ge Wang
Date: 2018-11-26 00:00:00
Tasks: Computed Tomography (CT), Diagnostic, Generative Adversarial Network

A major challenge in computed tomography (CT) is how to minimize patient radiation exposure without compromising image quality and diagnostic performance. The use of deep convolutional (Conv) neural networks for noise reduction in Low-Dose CT (LDCT) images has recently shown a great potential in this important application. In this paper, we present a highly efficient and effective neural network model for LDCT image noise reduction. Specifically, to capture local anatomical features we integrate Deep Convolutional Neural Networks (CNNs) and Skip connection layers for feature extraction. Also, we introduce parallelized $1\times 1$ CNN, called Network in Network, to lower the dimensionality of the output from the previous layer, achieving faster computational speed at less feature loss. To optimize the performance of the network, we adopt a Wasserstein generative adversarial network (WGAN) framework. Quantitative and qualitative comparisons demonstrate that our proposed network model can produce images with lower noise and more structural details than state-of-the-art noise-reduction methods.
CLEAR: A Dataset for Compositional Language and Elementary Acoustic Reasoning
Authors: Jerome Abdelnour, Giampiero Salvi, Jean Rouat
Date: 2018-11-26 00:00:00
Tasks: Acoustic Question Answering, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

We introduce the task of acoustic question answering (AQA) in the area of
acoustic reasoning. In this task an agent learns to answer questions on the
basis of acoustic context. In order to promote research in this area, we
propose a data generation paradigm adapted from CLEVR (Johnson et al. 2017). We
generate acoustic scenes by leveraging a bank elementary sounds. We also
provide a number of functional programs that can be used to compose questions
and answers that exploit the relationships between the attributes of the
elementary sounds in each scene. We provide AQA datasets of various sizes as
well as the data generation code. As a preliminary experiment to validate our
data, we report the accuracy of current state of the art visual question
answering models when they are applied to the AQA task without modifications.
Although there is a plethora of question answering tasks based on text, image
or video data, to our knowledge, we are the first to propose answering
questions directly on audio streams. We hope this contribution will facilitate
the development of research in the area.
Leveraging Filter Correlations for Deep Model Compression
Authors: Pravendra Singh, Vinay Kumar Verma, Piyush Rai, Vinay P. Namboodiri
Date: 2018-11-26 00:00:00
Tasks: model, Model Compression, object-detection, Object Detection

We present a filter correlation based model compression approach for deep convolutional neural networks. Our approach iteratively identifies pairs of filters with the largest pairwise correlations and drops one of the filters from each such pair. However, instead of discarding one of the filters from each such pair na\"{i}vely, the model is re-optimized to make the filters in these pairs maximally correlated, so that discarding one of the filters from the pair results in minimal information loss. Moreover, after discarding the filters in each round, we further finetune the model to recover from the potential small loss incurred by the compression. We evaluate our proposed approach using a comprehensive set of experiments and ablation studies. Our compression method yields state-of-the-art FLOPs compression rates on various benchmarks, such as LeNet-5, VGG-16, and ResNet-50,56, while still achieving excellent predictive performance for tasks such as object detection on benchmark datasets.
A deep neural network to enhance prediction of 1-year mortality using echocardiographic videos of the heart
Authors: Alvaro Ulloa, Linyuan Jing, Christopher W. Good, David P. vanMaanen, Sushravya Raghunath, Jonathan D Suever, Christopher D Nevius, Gregory J Wehner, Dustin Hartzel, Joseph B. Leader, Amro Alsaid, Aalpen A. Patel, H. Lester Kirchner, Marios S. Pattichis, Christopher M. Haggerty, Brandon K. Fornwalt
Date: 2018-11-26 00:00:00
Tasks: Mortality Prediction

Predicting future clinical events helps physicians guide appropriate intervention. Machine learning has tremendous promise to assist physicians with predictions based on the discovery of complex patterns from historical data, such as large, longitudinal electronic health records (EHR). This study is a first attempt to demonstrate such capabilities using raw echocardiographic videos of the heart. We show that a large dataset of 723,754 clinically-acquired echocardiographic videos (~45 million images) linked to longitudinal follow-up data in 27,028 patients can be used to train a deep neural network to predict 1-year mortality with good accuracy (area under the curve (AUC) in an independent test set = 0.839). Prediction accuracy was further improved by adding EHR data (AUC = 0.858). Finally, we demonstrate that the trained neural network was more accurate in mortality prediction than two expert cardiologists. These results highlight the potential of neural networks to add new power to clinical predictions.
Similarity-preserving Image-image Domain Adaptation for Person Re-identification
Authors: Weijian Deng, Liang Zheng, Qixiang Ye, Yi Yang, Jianbin Jiao
Date: 2018-11-26 00:00:00
Tasks: Domain Adaptation, Generative Adversarial Network, Person Re-Identification, Translation

This article studies the domain adaptation problem in person re-identification (re-ID) under a "learning via translation" framework, consisting of two components, 1) translating the labeled images from the source to the target domain in an unsupervised manner, 2) learning a re-ID model using the translated images. The objective is to preserve the underlying human identity information after image translation, so that translated images with labels are effective for feature learning on the target domain. To this end, we propose a similarity preserving generative adversarial network (SPGAN) and its end-to-end trainable version, eSPGAN. Both aiming at similarity preserving, SPGAN enforces this property by heuristic constraints, while eSPGAN does so by optimally facilitating the re-ID model learning. More specifically, SPGAN separately undertakes the two components in the "learning via translation" framework. It first preserves two types of unsupervised similarity, namely, self-similarity of an image before and after translation, and domain-dissimilarity of a translated source image and a target image. It then learns a re-ID model using existing networks. In comparison, eSPGAN seamlessly integrates image translation and re-ID model learning. During the end-to-end training of eSPGAN, re-ID learning guides image translation to preserve the underlying identity information of an image. Meanwhile, image translation improves re-ID learning by providing identity-preserving training samples of the target domain style. In the experiment, we show that identities of the fake images generated by SPGAN and eSPGAN are well preserved. Based on this, we report the new state-of-the-art domain adaptation results on two large-scale person re-ID datasets.
Challenges in the Automatic Analysis of Students' Diagnostic Reasoning
Authors: Claudia Schulz, Christian M. Meyer, Michael Sailer, Jan Kiesewetter, Elisabeth Bauer, Frank Fischer, Martin R. Fischer, Iryna Gurevych
Date: 2018-11-26 00:00:00
Tasks: Diagnostic

Diagnostic reasoning is a key component of many professions. To improve
students' diagnostic reasoning skills, educational psychologists analyse and
give feedback on epistemic activities used by these students while diagnosing,
in particular, hypothesis generation, evidence generation, evidence evaluation,
and drawing conclusions. However, this manual analysis is highly
time-consuming. We aim to enable the large-scale adoption of diagnostic
reasoning analysis and feedback by automating the epistemic activity
identification. We create the first corpus for this task, comprising diagnostic
reasoning self-explanations of students from two domains annotated with
epistemic activities. Based on insights from the corpus creation and the task's
characteristics, we discuss three challenges for the automatic identification
of epistemic activities using AI methods: the correct identification of
epistemic activity spans, the reliable distinction of similar epistemic
activities, and the detection of overlapping epistemic activities. We propose a
separate performance metric for each challenge and thus provide an evaluation
framework for future research. Indeed, our evaluation of various
state-of-the-art recurrent neural network architectures reveals that current
techniques fail to address some of these challenges.
Higher-order Projected Power Iterations for Scalable Multi-Matching
Authors: Florian Bernard, Johan Thunberg, Paul Swoboda, Christian Theobalt
Date: 2018-11-26 00:00:00

The matching of multiple objects (e.g. shapes or images) is a fundamental
problem in vision and graphics. In order to robustly handle ambiguities, noise
and repetitive patterns in challenging real-world settings, it is essential to
take geometric consistency between points into account. Computationally, the
multi-matching problem is difficult. It can be phrased as simultaneously
solving multiple (NP-hard) quadratic assignment problems (QAPs) that are
coupled via cycle-consistency constraints. The main limitations of existing
multi-matching methods are that they either ignore geometric consistency and
thus have limited robustness, or they are restricted to small-scale problems
due to their (relatively) high computational cost. We address these
shortcomings by introducing a Higher-order Projected Power Iteration method,
which is (i) efficient and scales to tens of thousands of points, (ii)
straightforward to implement, (iii) able to incorporate geometric consistency,
(iv) guarantees cycle-consistent multi-matchings, and (iv) comes with
theoretical convergence guarantees. Experimentally we show that our approach is
superior to existing methods.
Scene Categorization from Contours: Medial Axis Based Salience Measures
Authors: Morteza Rezanejad, Gabriel Downs, John Wilder, Dirk B. Walther, Allan Jepson, Sven Dickinson, Kaleem Siddiqi
Date: 2018-11-26 00:00:00
Proceeding: CVPR 2019 6
Tasks: General Classification, Scene Classification

The computer vision community has witnessed recent advances in scene
categorization from images, with the state-of-the art systems now achieving
impressive recognition rates on challenging benchmarks such as the Places365
dataset. Such systems have been trained on photographs which include color,
texture and shading cues. The geometry of shapes and surfaces, as conveyed by
scene contours, is not explicitly considered for this task. Remarkably, humans
can accurately recognize natural scenes from line drawings, which consist
solely of contour-based shape cues. Here we report the first computer vision
study on scene categorization of line drawings derived from popular databases
including an artist scene database, MIT67, and Places365. Specifically, we use
off-the-shelf pre-trained CNNs to perform scene classification given only
contour information as input and find performance levels well above chance. We
also show that medial-axis based contour salience methods can be used to select
more informative subsets of contour pixels and that the variation in CNN
classification performance on various choices for these subsets is
qualitatively similar to that observed in human performance. Moreover, when the
salience measures are used to weight the contours, as opposed to pruning them,
we find that these weights boost our CNN performance above that for unweighted
contour input. That is, the medial axis based salience weights appear to add
useful information that is not available when CNNs are trained to use contours
alone.
Predicting Language Recovery after Stroke with Convolutional Networks on Stitched MRI
Authors: Yusuf H. Roohani, Noor Sajid, Pranava Madhyastha, Cathy J. Price, Thomas M. H. Hope
Date: 2018-11-26 00:00:00

One third of stroke survivors have language difficulties. Emerging evidence
suggests that their likelihood of recovery depends mainly on the damage to
language centers. Thus previous research for predicting language recovery
post-stroke has focused on identifying damaged regions of the brain. In this
paper, we introduce a novel method where we only make use of stitched
2-dimensional cross-sections of raw MRI scans in a deep convolutional neural
network setup to predict language recovery post-stroke. Our results show: a)
the proposed model that only uses MRI scans has comparable performance to
models that are dependent on lesion specific information; b) the features
learned by our model are complementary to the lesion specific information and
the combination of both appear to outperform previously reported results in
similar settings. We further analyse the CNN model for understanding regions in
brain that are responsible for arriving at these predictions using gradient
based saliency maps. Our findings are in line with previous lesion studies.
Unsupervised 3D Shape Learning from Image Collections in the Wild
Authors: Attila Szabó, Paolo Favaro
Date: 2018-11-26 00:00:00
Tasks: Object

We present a method to learn the 3D surface of objects directly from a
collection of images. Previous work achieved this capability by exploiting
additional manual annotation, such as object pose, 3D surface templates,
temporal continuity of videos, manually selected landmarks, and
foreground/background masks. In contrast, our method does not make use of any
such annotation. Rather, it builds a generative model, a convolutional neural
network, which, given a noise vector sample, outputs the 3D surface and texture
of an object and a background image. These 3 components combined with an
additional random viewpoint vector are then fed to a differential renderer to
produce a view of the sampled object and background. Our general principle is
that if the output of the renderer, the generated image, is realistic, then its
input, the generated 3D and texture, should also be realistic. To achieve
realism, the generative model is trained adversarially against a discriminator
that tries to distinguish between the output of the renderer and real images
from the given data set. Moreover, our generative model can be paired with an
encoder and trained as an autoencoder, to automatically extract the 3D shape,
texture and pose of the object in an image. Our trained generative model and
encoder show promising results both on real and synthetic data, which
demonstrate for the first time that fully unsupervised 3D learning from image
collections is possible.
Deep Network Interpolation for Continuous Imagery Effect Transition
Authors: Xintao Wang, Ke Yu, Chao Dong, Xiaoou Tang, Chen Change Loy
Date: 2018-11-26 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Restoration, Image-to-Image Translation, Style Transfer, Translation

Deep convolutional neural network has demonstrated its capability of learning
a deterministic mapping for the desired imagery effect. However, the large
variety of user flavors motivates the possibility of continuous transition
among different output effects. Unlike existing methods that require a specific
design to achieve one particular transition (e.g., style transfer), we propose
a simple yet universal approach to attain a smooth control of diverse imagery
effects in many low-level vision tasks, including image restoration,
image-to-image translation, and style transfer. Specifically, our method,
namely Deep Network Interpolation (DNI), applies linear interpolation in the
parameter space of two or more correlated networks. A smooth control of imagery
effects can be achieved by tweaking the interpolation coefficients. In addition
to DNI and its broad applications, we also investigate the mechanism of network
interpolation from the perspective of learned filters.
Tracing in 2D to Reduce the Annotation Effort for 3D Deep Delineation
Authors: Mateusz Koziński, Agata Mosinska, Mathieu Salzmann, Pascal Fua
Date: 2018-11-26 00:00:00

The difficulty of obtaining annotations to build training databases still
slows down the adoption of recent deep learning approaches for biomedical image
analysis. In this paper, we show that we can train a Deep Net to perform 3D
volumetric delineation given only 2D annotations in Maximum Intensity
Projections (MIP). As a consequence, we can decrease the amount of time spent
annotating by a factor of two while maintaining similar performance.
  Our approach is inspired by space carving, a classical technique of
reconstructing complex 3D shapes from arbitrarily-positioned cameras. We will
demonstrate its effectiveness on 3D light microscopy images of neurons and
retinal blood vessels and on Magnetic Resonance Angiography (MRA) brain scans.
Deep Ensemble Tensor Factorization for Longitudinal Patient Trajectories Classification
Authors: Edward De Brouwer, Jaak Simm, Adam Arany, Yves Moreau
Date: 2018-11-26 00:00:00
Tasks: Classification, General Classification, Mortality Prediction, Time Series, Time Series Analysis

We present a generative approach to classify scarcely observed longitudinal
patient trajectories. The available time series are represented as tensors and
factorized using generative deep recurrent neural networks. The learned factors
represent the patient data in a compact way and can then be used in a
downstream classification task. For more robustness and accuracy in the
predictions, we used an ensemble of those deep generative models to mimic
Bayesian posterior sampling. We illustrate the performance of our architecture
on an intensive-care case study of in-hospital mortality prediction with 96
longitudinal measurement types measured across the first 48-hour from
admission. Our combination of generative and ensemble strategies achieves an
AUC of over 0.85, and outperforms the SAPS-II mortality score and GRU
baselines.