pwc-archive/papers-with-abstracts huggingface dataset:
Deep Reinforcement Learning-based Image Captioning with Embedding Reward
Authors: Zhou Ren, Xiaoyu Wang, Ning Zhang, Xutao Lv, Li-Jia Li
Date: 2017-04-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: Decision Making, Decoder, Deep Reinforcement Learning, Image Captioning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Image captioning is a challenging problem owing to the complexity in
understanding the image content and diverse ways of describing it in natural
language. Recent advances in deep neural networks have substantially improved
the performance of this task. Most state-of-the-art approaches follow an
encoder-decoder framework, which generates captions using a sequential
recurrent prediction model. However, in this paper, we introduce a novel
decision-making framework for image captioning. We utilize a "policy network"
and a "value network" to collaboratively generate captions. The policy network
serves as a local guidance by providing the confidence of predicting the next
word according to the current state. Additionally, the value network serves as
a global and lookahead guidance by evaluating all possible extensions of the
current state. In essence, it adjusts the goal of predicting the correct words
towards the goal of generating captions similar to the ground truth captions.
We train both networks using an actor-critic reinforcement learning model, with
a novel reward defined by visual-semantic embedding. Extensive experiments and
analyses on the Microsoft COCO dataset show that the proposed framework
outperforms state-of-the-art approaches across different evaluation metrics.
What's in a Question: Using Visual Questions as a Form of Supervision
Authors: Siddha Ganju, Olga Russakovsky, Abhinav Gupta
Date: 2017-04-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: Data Augmentation, Form, Visual Question Answering, Visual Question Answering (VQA)

Collecting fully annotated image datasets is challenging and expensive. Many
types of weak supervision have been explored: weak manual annotations, web
search results, temporal continuity, ambient sound and others. We focus on one
particular unexplored mode: visual questions that are asked about images. The
key observation that inspires our work is that the question itself provides
useful information about the image (even without the answer being available).
For instance, the question "what is the breed of the dog?" informs the AI that
the animal in the scene is a dog and that there is only one dog present. We
make three contributions: (1) providing an extensive qualitative and
quantitative analysis of the information contained in human visual questions,
(2) proposing two simple but surprisingly effective modifications to the
standard visual question answering models that allow them to make use of weak
supervision in the form of unanswered questions associated with images and (3)
demonstrating that a simple data augmentation strategy inspired by our insights
results in a 7.1% improvement on the standard VQA benchmark.
A Sub-Quadratic Exact Medoid Algorithm
Authors: James Newling, François Fleuret
Date: 2016-05-23 00:00:00

We present a new algorithm, trimed, for obtaining the medoid of a set, that
is the element of the set which minimises the mean distance to all other
elements. The algorithm is shown to have, under certain assumptions, expected
run time O(N^(3/2)) in R^d where N is the set size, making it the first
sub-quadratic exact medoid algorithm for d>1. Experiments show that it performs
very well on spatial network data, frequently requiring two orders of magnitude
fewer distance calculations than state-of-the-art approximate algorithms. As an
application, we show how trimed can be used as a component in an accelerated
K-medoids algorithm, and then how it can be relaxed to obtain further
computational gains with only a minor loss in cluster quality.
Gaze Embeddings for Zero-Shot Image Classification
Authors: Nour Karessli, Zeynep Akata, Bernt Schiele, Andreas Bulling
Date: 2016-11-28 00:00:00
Proceeding: CVPR 2017 7
Tasks: Classification, Fine-Grained Image Classification, General Classification, image-classification, Image Classification, Zero-Shot Image Classification

Zero-shot image classification using auxiliary information, such as
attributes describing discriminative object properties, requires time-consuming
annotation by domain experts. We instead propose a method that relies on human
gaze as auxiliary information, exploiting that even non-expert users have a
natural ability to judge class membership. We present a data collection
paradigm that involves a discrimination task to increase the information
content obtained from gaze data. Our method extracts discriminative descriptors
from the data and learns a compatibility function between image and gaze using
three novel gaze embeddings: Gaze Histograms (GH), Gaze Features with Grid
(GFG) and Gaze Features with Sequence (GFS). We introduce two new
gaze-annotated datasets for fine-grained image classification and show that
human gaze data is indeed class discriminative, provides a competitive
alternative to expert-annotated attributes, and outperforms other baselines for
zero-shot image classification.
A Brief Introduction to the Temporal Group LASSO and its Potential Applications in Healthcare
Authors: Diego Saldana Miranda
Date: 2017-04-07 00:00:00
Tasks: regression

The Temporal Group LASSO is an example of a multi-task, regularized
regression approach for the prediction of response variables that vary over
time. The aim of this work is to introduce the reader to the concepts behind
the Temporal Group LASSO and its related methods, as well as to the type of
potential applications in a healthcare setting that the method has. We argue
that the method is attractive because of its ability to reduce overfitting,
select predictors, learn smooth effect patterns over time, and finally, its
simplicity
Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark
Authors: Timo Hackel, Nikolay Savinov, Lubor Ladicky, Jan D. Wegner, Konrad Schindler, Marc Pollefeys
Date: 2017-04-12 00:00:00
Tasks: 3D Point Cloud Classification, Classification, Deep Learning, General Classification, object-detection, Object Detection, Point Cloud Classification, Semantic Segmentation

This paper presents a new 3D point cloud classification benchmark data set
with over four billion manually labelled points, meant as input for data-hungry
(deep) learning methods. We also discuss first submissions to the benchmark
that use deep convolutional neural networks (CNNs) as a work horse, which
already show remarkable performance improvements over state-of-the-art. CNNs
have become the de-facto standard for many tasks in computer vision and machine
learning like semantic segmentation or object detection in images, but have no
yet led to a true breakthrough for 3D point cloud labelling tasks due to lack
of training data. With the massive data set presented in this paper, we aim at
closing this data gap to help unleash the full potential of deep learning
methods for 3D labelling tasks. Our semantic3D.net data set consists of dense
point clouds acquired with static terrestrial laser scanners. It contains 8
semantic classes and covers a wide range of urban outdoor scenes: churches,
streets, railroad tracks, squares, villages, soccer fields and castles. We
describe our labelling interface and show that our data set provides more dense
and complete point clouds with much higher overall number of labelled points
compared to those already available to the research community. We further
provide baseline method descriptions and comparison between methods submitted
to our online system. We hope semantic3D.net will pave the way for deep
learning methods in 3D point cloud labelling to learn richer, more general 3D
representations, and first submissions after only a few months indicate that
this might indeed be the case.
Determining Song Similarity via Machine Learning Techniques and Tagging Information
Authors: Renato L. F. Cunha, Evandro Caldeira, Luciana Fujii
Date: 2017-04-12 00:00:00
Tasks: BIG-bench Machine Learning, Recommendation Systems, regression

The task of determining item similarity is a crucial one in a recommender
system. This constitutes the base upon which the recommender system will work
to determine which items are more likely to be enjoyed by a user, resulting in
more user engagement. In this paper we tackle the problem of determining song
similarity based solely on song metadata (such as the performer, and song
title) and on tags contributed by users. We evaluate our approach under a
series of different machine learning algorithms. We conclude that tf-idf
achieves better results than Word2Vec to model the dataset to feature vectors.
We also conclude that k-NN models have better performance than SVMs and Linear
Regression for this problem.
Improved Sparse Low-Rank Matrix Estimation
Authors: Ankit Parekh, Ivan W. Selesnick
Date: 2016-04-29 00:00:00
Tasks: Denoising

We address the problem of estimating a sparse low-rank matrix from its noisy
observation. We propose an objective function consisting of a data-fidelity
term and two parameterized non-convex penalty functions. Further, we show how
to set the parameters of the non-convex penalty functions, in order to ensure
that the objective function is strictly convex. The proposed objective function
better estimates sparse low-rank matrices than a convex method which utilizes
the sum of the nuclear norm and the $\ell_1$ norm. We derive an algorithm (as
an instance of ADMM) to solve the proposed problem, and guarantee its
convergence provided the scalar augmented Lagrangian parameter is set
appropriately. We demonstrate the proposed method for denoising an audio signal
and an adjacency matrix representing protein interactions in the `Escherichia
coli' bacteria.
Connecting Look and Feel: Associating the visual and tactile properties of physical materials
Authors: Wenzhen Yuan, Shaoxiong Wang, Siyuan Dong, Edward Adelson
Date: 2017-04-12 00:00:00
Proceeding: CVPR 2017 7

For machines to interact with the physical world, they must understand the
physical properties of objects and materials they encounter. We use fabrics as
an example of a deformable material with a rich set of mechanical properties. A
thin flexible fabric, when draped, tends to look different from a heavy stiff
fabric. It also feels different when touched. Using a collection of 118 fabric
sample, we captured color and depth images of draped fabrics along with tactile
data from a high resolution touch sensor. We then sought to associate the
information from vision and touch by jointly training CNNs across the three
modalities. Through the CNN, each input, regardless of the modality, generates
an embedding vector that records the fabric's physical property. By comparing
the embeddings, our system is able to look at a fabric image and predict how it
will feel, and vice versa. We also show that a system jointly trained on vision
and touch data can outperform a similar system trained only on visual data when
tested purely with visual inputs.
Fast Learning and Prediction for Object Detection using Whitened CNN Features
Authors: Björn Barz, Erik Rodner, Christoph Käding, Joachim Denzler
Date: 2017-04-10 00:00:00
Tasks: Feature Engineering, object-detection, Object Detection, Real-Time Object Detection

We combine features extracted from pre-trained convolutional neural networks
(CNNs) with the fast, linear Exemplar-LDA classifier to get the advantages of
both: the high detection performance of CNNs, automatic feature engineering,
fast model learning from few training samples and efficient sliding-window
detection. The Adaptive Real-Time Object Detection System (ARTOS) has been
refactored broadly to be used in combination with Caffe for the experimental
studies reported in this work.
Intrinsic Light Field Images
Authors: Elena Garces, Jose I. Echevarria, Wen Zhang, Hongzhi Wu, Kun Zhou, Diego Gutierrez
Date: 2016-08-15 00:00:00

We present a method to automatically decompose a light field into its
intrinsic shading and albedo components. Contrary to previous work targeted to
2D single images and videos, a light field is a 4D structure that captures
non-integrated incoming radiance over a discrete angular domain. This higher
dimensionality of the problem renders previous state-of-the-art algorithms
impractical either due to their cost of processing a single 2D slice, or their
inability to enforce proper coherence in additional dimensions. We propose a
new decomposition algorithm that jointly optimizes the whole light field data
for proper angular coherence. For efficiency, we extend Retinex theory, working
on the gradient domain, where new albedo and occlusion terms are introduced.
Results show our method provides 4D intrinsic decompositions difficult to
achieve with previous state-of-the-art algorithms. We further provide a
comprehensive analysis and comparisons with existing intrinsic image/video
decomposition methods on light field images.
Stacked Generative Adversarial Networks
Authors: Xun Huang, Yixuan Li, Omid Poursaeed, John Hopcroft, Serge Belongie
Date: 2016-12-13 00:00:00
Proceeding: CVPR 2017 7
Tasks: Conditional Image Generation

In this paper, we propose a novel generative model named Stacked Generative
Adversarial Networks (SGAN), which is trained to invert the hierarchical
representations of a bottom-up discriminative network. Our model consists of a
top-down stack of GANs, each learned to generate lower-level representations
conditioned on higher-level representations. A representation discriminator is
introduced at each feature hierarchy to encourage the representation manifold
of the generator to align with that of the bottom-up discriminative network,
leveraging the powerful discriminative representations to guide the generative
model. In addition, we introduce a conditional loss that encourages the use of
conditional information from the layer above, and a novel entropy loss that
maximizes a variational lower bound on the conditional entropy of generator
outputs. We first train each stack independently, and then train the whole
model end-to-end. Unlike the original GAN that uses a single noise vector to
represent all the variations, our SGAN decomposes variations into multiple
levels and gradually resolves uncertainties in the top-down generative process.
Based on visual inspection, Inception scores and visual Turing test, we
demonstrate that SGAN is able to generate images of much higher quality than
GANs without stacking.
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Authors: Clément Godard, Oisin Mac Aodha, Gabriel J. Brostow
Date: 2016-09-13 00:00:00
Proceeding: CVPR 2017 7
Tasks: Depth Estimation, Depth Prediction, Image Reconstruction, Monocular Depth Estimation, Unsupervised Monocular Depth Estimation

Learning based methods have shown very promising results for the task of
depth estimation in single images. However, most existing approaches treat
depth prediction as a supervised regression problem and as a result, require
vast quantities of corresponding ground truth depth data for training. Just
recording quality depth data in a range of environments is a challenging
problem. In this paper, we innovate beyond existing approaches, replacing the
use of explicit depth data during training with easier-to-obtain binocular
stereo footage.
  We propose a novel training objective that enables our convolutional neural
network to learn to perform single image depth estimation, despite the absence
of ground truth depth data. Exploiting epipolar geometry constraints, we
generate disparity images by training our network with an image reconstruction
loss. We show that solving for image reconstruction alone results in poor
quality depth images. To overcome this problem, we propose a novel training
loss that enforces consistency between the disparities produced relative to
both the left and right images, leading to improved performance and robustness
compared to existing approaches. Our method produces state of the art results
for monocular depth estimation on the KITTI driving dataset, even outperforming
supervised methods that have been trained with ground truth depth.
Spatiotemporal Networks for Video Emotion Recognition
Authors: Lijie Fan, Yunjie Ke
Date: 2017-04-03 00:00:00
Tasks: Classification, Emotion Recognition, General Classification, Video Emotion Recognition

Our experiment adapts several popular deep learning methods as well as some
traditional methods on the problem of video emotion recognition. In our
experiment, we use the CNN-LSTM architecture for visual information extraction
and classification and utilize traditional methods such as for audio feature
classification. For multimodal fusion, we use the traditional Support Vector
Machine. Our experiment yields a good result on the AFEW 6.0 Dataset.
Parallelized Kendall's Tau Coefficient Computation via SIMD Vectorized Sorting On Many-Integrated-Core Processors
Authors: Yongchao Liu, Tony Pan, Oded Green, Srinivas Aluru
Date: 2017-04-12 00:00:00
Tasks: Distributed Computing

Pairwise association measure is an important operation in data analytics.
Kendall's tau coefficient is one widely used correlation coefficient
identifying non-linear relationships between ordinal variables. In this paper,
we investigated a parallel algorithm accelerating all-pairs Kendall's tau
coefficient computation via single instruction multiple data (SIMD) vectorized
sorting on Intel Xeon Phis by taking advantage of many processing cores and
512-bit SIMD vector instructions. To facilitate workload balancing and overcome
on-chip memory limitation, we proposed a generic framework for symmetric
all-pairs computation by building provable bijective functions between job
identifier and coordinate space. Performance evaluation demonstrated that our
algorithm on one 5110P Phi achieves two orders-of-magnitude speedups over
16-threaded MATLAB and three orders-of-magnitude speedups over sequential R,
both running on high-end CPUs. Besides, our algorithm exhibited rather good
distributed computing scalability with respect to number of Phis. Source code
and datasets are publicly available at http://lightpcc.sourceforge.net.
Interacting Particle Markov Chain Monte Carlo
Authors: Tom Rainforth, Christian A. Naesseth, Fredrik Lindsten, Brooks Paige, Jan-Willem van de Meent, Arnaud Doucet, Frank Wood
Date: 2016-02-16 00:00:00

We introduce interacting particle Markov chain Monte Carlo (iPMCMC), a PMCMC
method based on an interacting pool of standard and conditional sequential
Monte Carlo samplers. Like related methods, iPMCMC is a Markov chain Monte
Carlo sampler on an extended space. We present empirical results that show
significant improvements in mixing rates relative to both non-interacting PMCMC
samplers, and a single PMCMC sampler with an equivalent memory and
computational budget. An additional advantage of the iPMCMC method is that it
is suitable for distributed and multi-core architectures.
Unsupervised part learning for visual recognition
Authors: Ronan Sicre, Yannis Avrithis, Ewa Kijak, Frederic Jurie
Date: 2017-04-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: Classification, General Classification, image-classification, Image Classification, Image Retrieval, Retrieval

Part-based image classification aims at representing categories by small sets
of learned discriminative parts, upon which an image representation is built.
Considered as a promising avenue a decade ago, this direction has been
neglected since the advent of deep neural networks. In this context, this paper
brings two contributions: first, it shows that despite the recent success of
end-to-end holistic models, explicit part learning can boosts classification
performance. Second, this work proceeds one step further than recent part-based
models (PBM), focusing on how to learn parts without using any labeled data.
Instead of learning a set of parts per class, as generally done in the PBM
literature, the proposed approach both constructs a partition of a given set of
images into visually similar groups, and subsequently learn a set of
discriminative parts per group in a fully unsupervised fashion. This strategy
opens the door to the use of PBM in new applications for which the notion of
image categories is irrelevant, such as instance-based image retrieval, for
example. We experimentally show that our learned parts can help building
efficient image representations, for classification as well as for indexing
tasks, resulting in performance superior to holistic state-of-the art Deep
Convolutional Neural Networks (DCNN) encoding.
MRA - Proof of Concept of a Multilingual Report Annotator Web Application
Authors: Luís Campos, Francisco Couto
Date: 2017-04-06 00:00:00

MRA (Multilingual Report Annotator) is a web application that translates
Radiology text and annotates it with RadLex terms. Its goal is to explore the
solution of translating non-English Radiology reports as a way to solve the
problem of most of the Text Mining tools being developed for English. In this
brief paper we explain the language barrier problem and shortly describe the
application. MRA can be found at https://github.com/lasigeBioTM/MRA .
Function Driven Diffusion for Personalized Counterfactual Inference
Authors: Alexander Cloninger
Date: 2016-10-31 00:00:00
Tasks: counterfactual, Counterfactual Inference

We consider the problem of constructing diffusion operators high dimensional
data $X$ to address counterfactual functions $F$, such as individualized
treatment effectiveness. We propose and construct a new diffusion metric $K_F$
that captures both the local geometry of $X$ and the directions of variance of
$F$. The resulting diffusion metric is then used to define a localized
filtration of $F$ and answer counterfactual questions pointwise, particularly
in situations such as drug trials where an individual patient's outcomes cannot
be studied long term both taking and not taking a medication. We validate the
model on synthetic and real world clinical trials, and create individualized
notions of benefit from treatment.
Deep-FExt: Deep Feature Extraction for Vessel Segmentation and Centerline Prediction
Authors: Giles Tetteh, Markus Rempfler, Bjoern H. Menze, Claus Zimmer
Date: 2017-04-12 00:00:00
Tasks: Classification, General Classification

Feature extraction is a very crucial task in image and pixel (voxel)
classification and regression in biomedical image modeling. In this work we
present a machine learning based feature extraction scheme based on inception
models for pixel classification tasks. We extract features under multi-scale
and multi-layer schemes through convolutional operators. Layers of Fully
Convolutional Network are later stacked on this feature extraction layers and
trained end-to-end for the purpose of classification. We test our model on the
DRIVE and STARE public data sets for the purpose of segmentation and centerline
detection and it out performs most existing hand crafted or deterministic
feature schemes found in literature. We achieve an average maximum Dice of 0.85
on the DRIVE data set which out performs the scores from the second human
annotator of this data set. We also achieve an average maximum Dice of 0.85 and
kappa of 0.84 on the STARE data set. Though these datasets are mainly 2-D we
also propose ways of extending this feature extraction scheme to handle 3-D
datasets.