pwc-archive/papers-with-abstracts huggingface dataset:
Least Squares Generative Adversarial Networks
Authors: Xudong Mao, Qing Li, Haoran Xie, Raymond Y. K. Lau, Zhen Wang, Stephen Paul Smolley
Date: 2016-11-13 00:00:00
Proceeding: ICCV 2017 10

Unsupervised learning with generative adversarial networks (GANs) has proven
hugely successful. Regular GANs hypothesize the discriminator as a classifier
with the sigmoid cross entropy loss function. However, we found that this loss
function may lead to the vanishing gradients problem during the learning
process. To overcome such a problem, we propose in this paper the Least Squares
Generative Adversarial Networks (LSGANs) which adopt the least squares loss
function for the discriminator. We show that minimizing the objective function
of LSGAN yields minimizing the Pearson $\chi^2$ divergence. There are two
benefits of LSGANs over regular GANs. First, LSGANs are able to generate higher
quality images than regular GANs. Second, LSGANs perform more stable during the
learning process. We evaluate LSGANs on five scene datasets and the
experimental results show that the images generated by LSGANs are of better
quality than the ones generated by regular GANs. We also conduct two comparison
experiments between LSGANs and regular GANs to illustrate the stability of
LSGANs.
Cross-modal Deep Metric Learning with Multi-task Regularization
Authors: Xin Huang, Yuxin Peng
Date: 2017-03-21 00:00:00
Tasks: Cross-Modal Retrieval, Metric Learning, Multi-Task Learning, Retrieval, Semantic Similarity, Semantic Textual Similarity

DNN-based cross-modal retrieval has become a research hotspot, by which users
can search results across various modalities like image and text. However,
existing methods mainly focus on the pairwise correlation and reconstruction
error of labeled data. They ignore the semantically similar and dissimilar
constraints between different modalities, and cannot take advantage of
unlabeled data. This paper proposes Cross-modal Deep Metric Learning with
Multi-task Regularization (CDMLMR), which integrates quadruplet ranking loss
and semi-supervised contrastive loss for modeling cross-modal semantic
similarity in a unified multi-task learning architecture. The quadruplet
ranking loss can model the semantically similar and dissimilar constraints to
preserve cross-modal relative similarity ranking information. The
semi-supervised contrastive loss is able to maximize the semantic similarity on
both labeled and unlabeled data. Compared to the existing methods, CDMLMR
exploits not only the similarity ranking information but also unlabeled
cross-modal data, and thus boosts cross-modal retrieval accuracy.
Supporting Navigation of Outdoor Shopping Complexes for Visually-impaired Users through Multi-modal Data Fusion
Authors: Archana Paladugu, Parag S. Chandakkar, Peng Zhang, Baoxin Li
Date: 2017-04-05 00:00:00
Tasks: Navigate

Outdoor shopping complexes (OSC) are extremely difficult for people with
visual impairment to navigate. Existing GPS devices are mostly designed for
roadside navigation and seldom transition well into an OSC-like setting. We
report our study on the challenges faced by a blind person in navigating OSC
through developing a new mobile application named iExplore. We first report an
exploratory study aiming at deriving specific design principles for building
this system by learning the unique challenges of the problem. Then we present a
methodology that can be used to derive the necessary information for the
development of iExplore, followed by experimental validation of the technology
by a group of visually impaired users in a local outdoor shopping center. User
feedback and other experiments suggest that iExplore, while at its very initial
phase, has the potential of filling a practical gap in existing assistive
technologies for the visually impaired.
Geometry of Factored Nuclear Norm Regularization
Authors: Qiuwei Li, Zhihui Zhu, Gongguo Tang
Date: 2017-04-05 00:00:00

This work investigates the geometry of a nonconvex reformulation of
minimizing a general convex loss function $f(X)$ regularized by the matrix
nuclear norm $\|X\|_*$. Nuclear-norm regularized matrix inverse problems are at
the heart of many applications in machine learning, signal processing, and
control. The statistical performance of nuclear norm regularization has been
studied extensively in literature using convex analysis techniques. Despite its
optimal performance, the resulting optimization has high computational
complexity when solved using standard or even tailored fast convex solvers. To
develop faster and more scalable algorithms, we follow the proposal of
Burer-Monteiro to factor the matrix variable $X$ into the product of two
smaller rectangular matrices $X=UV^T$ and also replace the nuclear norm
$\|X\|_*$ with $(\|U\|_F^2+\|V\|_F^2)/2$. In spite of the nonconvexity of the
factored formulation, we prove that when the convex loss function $f(X)$ is
$(2r,4r)$-restricted well-conditioned, each critical point of the factored
problem either corresponds to the optimal solution $X^\star$ of the original
convex optimization or is a strict saddle point where the Hessian matrix has a
strictly negative eigenvalue. Such a geometric structure of the factored
formulation allows many local search algorithms to converge to the global
optimum with random initializations.
Classification of Diabetic Retinopathy Images Using Multi-Class Multiple-Instance Learning Based on Color Correlogram Features
Authors: Ragav Venkatesan, Parag S. Chandakkar, Baoxin Li
Date: 2017-04-05 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Multiple Instance Learning

All people with diabetes have the risk of developing diabetic retinopathy
(DR), a vision-threatening complication. Early detection and timely treatment
can reduce the occurrence of blindness due to DR. Computer-aided diagnosis has
the potential benefit of improving the accuracy and speed in DR detection. This
study is concerned with automatic classification of images with microaneurysm
(MA) and neovascularization (NV), two important DR clinical findings. Together
with normal images, this presents a 3-class classification problem. We propose
a modified color auto-correlogram feature (AutoCC) with low dimensionality that
is spectrally tuned towards DR images. Recognizing the fact that the images
with or without MA or NV are generally different only in small, localized
regions, we propose to employ a multi-class, multiple-instance learning
framework for performing the classification task using the proposed feature.
Extensive experiments including comparison with a few state-of-art image
classification approaches have been performed and the results suggest that the
proposed approach is promising as it outperforms other methods by a large
margin.
Investigating Human Factors in Image Forgery Detection
Authors: Parag S. Chandakkar, Baoxin Li
Date: 2017-04-05 00:00:00
Tasks: Image Forgery Detection

In today's age of internet and social media, one can find an enormous volume
of forged images on-line. These images have been used in the past to convey
falsified information and achieve harmful intentions. The spread and the effect
of the social media only makes this problem more severe. While creating forged
images has become easier due to software advancements, there is no automated
algorithm which can reliably detect forgery.
  Image forgery detection can be seen as a subset of image understanding
problem. Human performance is still the gold-standard for these type of
problems when compared to existing state-of-art automated algorithms. We
conduct a subjective evaluation test with the aid of eye-tracker to investigate
into human factors associated with this problem. We compare the performance of
an automated algorithm and humans for forgery detection problem. We also
develop an algorithm which uses the data from the evaluation test to predict
the difficulty-level of an image (the difficulty-level of an image here denotes
how difficult it is for humans to detect forgery in an image. Terms such as
"Easy/difficult image" will be used in the same context). The experimental
results presented in this paper should facilitate development of better
algorithms in the future.
Improving Vision-based Self-positioning in Intelligent Transportation Systems via Integrated Lane and Vehicle Detection
Authors: Parag S. Chandakkar, Yilin Wang, Baoxin Li
Date: 2017-04-05 00:00:00
Tasks: Computational Efficiency, Density Estimation, Position, vehicle detection

Traffic congestion is a widespread problem. Dynamic traffic routing systems
and congestion pricing are getting importance in recent research. Lane
prediction and vehicle density estimation is an important component of such
systems. We introduce a novel problem of vehicle self-positioning which
involves predicting the number of lanes on the road and vehicle's position in
those lanes using videos captured by a dashboard camera. We propose an
integrated closed-loop approach where we use the presence of vehicles to aid
the task of self-positioning and vice-versa. To incorporate multiple factors
and high-level semantic knowledge into the solution, we formulate this problem
as a Bayesian framework. In the framework, the number of lanes, the vehicle's
position in those lanes and the presence of other vehicles are considered as
parameters. We also propose a bounding box selection scheme to reduce the
number of false detections and increase the computational efficiency. We show
that the number of box proposals decreases by a factor of 6 using the selection
approach. It also results in large reduction in the number of false detections.
The entire approach is tested on real-world videos and is found to give
acceptable results.
Linear Additive Markov Processes
Authors: Ravi Kumar, Maithra Raghu, Tamas Sarlos, Andrew Tomkins
Date: 2017-04-05 00:00:00

We introduce LAMP: the Linear Additive Markov Process. Transitions in LAMP
may be influenced by states visited in the distant history of the process, but
unlike higher-order Markov processes, LAMP retains an efficient
parametrization. LAMP also allows the specific dependence on history to be
learned efficiently from data. We characterize some theoretical properties of
LAMP, including its steady-state and mixing time. We then give an algorithm
based on alternating minimization to learn LAMP models from data. Finally, we
perform a series of real-world experiments to show that LAMP is more powerful
than first-order Markov processes, and even holds its own against deep
sequential models (LSTMs) with a negligible increase in parameter complexity.
Relative Learning from Web Images for Content-adaptive Enhancement
Authors: Parag S. Chandakkar, Qiongjie Tian, Baoxin Li
Date: 2017-04-05 00:00:00
Tasks: Image Enhancement

Personalized and content-adaptive image enhancement can find many
applications in the age of social media and mobile computing. This paper
presents a relative-learning-based approach, which, unlike previous methods,
does not require matching original and enhanced images for training. This
allows the use of massive online photo collections to train a ranking model for
improved enhancement. We first propose a multi-level ranking model, which is
learned from only relatively-labeled inputs that are automatically crawled.
Then we design a novel parameter sampling scheme under this model to generate
the desired enhancement parameters for a new image. For evaluation, we first
verify the effectiveness and the generalization abilities of our approach,
using images that have been enhanced/labeled by experts. Then we carry out
subjective tests, which show that users prefer images enhanced by our approach
over other existing methods.
A Structured Approach to Predicting Image Enhancement Parameters
Authors: Parag S. Chandakkar, Baoxin Li
Date: 2017-04-05 00:00:00
Tasks: Image Enhancement, Parameter Prediction, Structured Prediction

Social networking on mobile devices has become a commonplace of everyday
life. In addition, photo capturing process has become trivial due to the
advances in mobile imaging. Hence people capture a lot of photos everyday and
they want them to be visually-attractive. This has given rise to automated,
one-touch enhancement tools. However, the inability of those tools to provide
personalized and content-adaptive enhancement has paved way for machine-learned
methods to do the same. The existing typical machine-learned methods
heuristically (e.g. kNN-search) predict the enhancement parameters for a new
image by relating the image to a set of similar training images. These
heuristic methods need constant interaction with the training images which
makes the parameter prediction sub-optimal and computationally expensive at
test time which is undesired. This paper presents a novel approach to
predicting the enhancement parameters given a new image using only its
features, without using any training images. We propose to model the
interaction between the image features and its corresponding enhancement
parameters using the matrix factorization (MF) principles. We also propose a
way to integrate the image features in the MF formulation. We show that our
approach outperforms heuristic approaches as well as recent approaches in MF
and structured prediction on synthetic as well as real-world data of image
enhancement.
ParaGraphE: A Library for Parallel Knowledge Graph Embedding
Authors: Xiao-Fan Niu, Wu-Jun Li
Date: 2017-03-16 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs

Knowledge graph embedding aims at translating the knowledge graph into
numerical representations by transforming the entities and relations into
continuous low-dimensional vectors. Recently, many methods [1, 5, 3, 2, 6] have
been proposed to deal with this problem, but existing single-thread
implementations of them are time-consuming for large-scale knowledge graphs.
Here, we design a unified parallel framework to parallelize these methods,
which achieves a significant time reduction without influencing the accuracy.
We name our framework as ParaGraphE, which provides a library for parallel
knowledge graph embedding. The source code can be downloaded from
https://github.com/LIBBLE/LIBBLE-MultiThread/tree/master/ParaGraphE .
A Computational Approach to Relative Aesthetics
Authors: Parag S. Chandakkar, Vijetha Gattupalli, Baoxin Li
Date: 2017-04-05 00:00:00
Tasks: Binary Classification, General Classification, Image Retrieval, Retrieval

Computational visual aesthetics has recently become an active research area.
Existing state-of-art methods formulate this as a binary classification task
where a given image is predicted to be beautiful or not. In many applications
such as image retrieval and enhancement, it is more important to rank images
based on their aesthetic quality instead of binary-categorizing them.
Furthermore, in such applications, it may be possible that all images belong to
the same category. Hence determining the aesthetic ranking of the images is
more appropriate. To this end, we formulate a novel problem of ranking images
with respect to their aesthetic quality. We construct a new dataset of image
pairs with relative labels by carefully selecting images from the popular AVA
dataset. Unlike in aesthetics classification, there is no single threshold
which would determine the ranking order of the images across our entire
dataset. We propose a deep neural network based approach that is trained on
image pairs by incorporating principles from relative learning. Results show
that such relative training procedure allows our network to rank the images
with a higher accuracy than a state-of-art network trained on the same set of
images using binary labels.
Estimation of Tissue Microstructure Using a Deep Network Inspired by a Sparse Reconstruction Framework
Authors: Chuyang Ye
Date: 2017-04-05 00:00:00

Diffusion magnetic resonance imaging (dMRI) provides a unique tool for
noninvasively probing the microstructure of the neuronal tissue. The NODDI
model has been a popular approach to the estimation of tissue microstructure in
many neuroscience studies. It represents the diffusion signals with three types
of diffusion in tissue: intra-cellular, extra-cellular, and cerebrospinal fluid
compartments. However, the original NODDI method uses a computationally
expensive procedure to fit the model and could require a large number of
diffusion gradients for accurate microstructure estimation, which may be
impractical for clinical use. Therefore, efforts have been devoted to efficient
and accurate NODDI microstructure estimation with a reduced number of diffusion
gradients. In this work, we propose a deep network based approach to the NODDI
microstructure estimation, which is named Microstructure Estimation using a
Deep Network (MEDN). Motivated by the AMICO algorithm which accelerates the
computation of NODDI parameters, we formulate the microstructure estimation
problem in a dictionary-based framework. The proposed network comprises two
cascaded stages. The first stage resembles the solution to a dictionary-based
sparse reconstruction problem and the second stage computes the final
microstructure using the output of the first stage. The weights in the two
stages are jointly learned from training data, which is obtained from training
dMRI scans with diffusion gradients that densely sample the q-space. The
proposed method was applied to brain dMRI scans, where two shells each with 30
gradient directions (60 diffusion gradients in total) were used. Estimation
accuracy with respect to the gold standard was measured and the results
demonstrate that MEDN outperforms the competing algorithms.
Joint Regression and Ranking for Image Enhancement
Authors: Parag S. Chandakkar, Baoxin Li
Date: 2017-04-05 00:00:00
Tasks: Image Enhancement, regression

Research on automated image enhancement has gained momentum in recent years,
partially due to the need for easy-to-use tools for enhancing pictures captured
by ubiquitous cameras on mobile devices. Many of the existing leading methods
employ machine-learning-based techniques, by which some enhancement parameters
for a given image are found by relating the image to the training images with
known enhancement parameters. While knowing the structure of the parameter
space can facilitate search for the optimal solution, none of the existing
methods has explicitly modeled and learned that structure. This paper presents
an end-to-end, novel joint regression and ranking approach to model the
interaction between desired enhancement parameters and images to be processed,
employing a Gaussian process (GP). GP allows searching for ideal parameters
using only the image features. The model naturally leads to a ranking technique
for comparing images in the induced feature space. Comparative evaluation using
the ground-truth based on the MIT-Adobe FiveK dataset plus subjective tests on
an additional data-set were used to demonstrate the effectiveness of the
proposed approach.
A Brownian Motion Model and Extreme Belief Machine for Modeling Sensor Data Measurements
Authors: Robert A. Murphy
Date: 2017-04-01 00:00:00

As the title suggests, we will describe (and justify through the presentation
of some of the relevant mathematics) prediction methodologies for sensor
measurements. This exposition will mainly be concerned with the mathematics
related to modeling the sensor measurements.
Two Stream LSTM: A Deep Fusion Framework for Human Action Recognition
Authors: Harshala Gammulle, Simon Denman, Sridha Sridharan, Clinton Fookes
Date: 2017-04-04 00:00:00
Tasks: Action Recognition, Temporal Action Localization, Vocal Bursts Valence Prediction

In this paper we address the problem of human action recognition from video
sequences. Inspired by the exemplary results obtained via automatic feature
learning and deep learning approaches in computer vision, we focus our
attention towards learning salient spatial features via a convolutional neural
network (CNN) and then map their temporal relationship with the aid of
Long-Short-Term-Memory (LSTM) networks. Our contribution in this paper is a
deep fusion framework that more effectively exploits spatial features from CNNs
with temporal features from LSTM models. We also extensively evaluate their
strengths and weaknesses. We find that by combining both the sets of features,
the fully connected features effectively act as an attention mechanism to
direct the LSTM to interesting parts of the convolutional feature sequence. The
significance of our fusion method is its simplicity and effectiveness compared
to other state-of-the-art methods. The evaluation results demonstrate that this
hierarchical multi stream fusion method has higher performance compared to
single stream mapping methods allowing it to achieve high accuracy
outperforming current state-of-the-art methods in three widely used databases:
UCF11, UCFSports, jHMDB.
Deep Information Propagation
Authors: Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein
Date: 2016-11-04 00:00:00

We study the behavior of untrained neural networks whose weights and biases
are randomly distributed using mean field theory. We show the existence of
depth scales that naturally limit the maximum depth of signal propagation
through these random networks. Our main practical result is to show that random
networks may be trained precisely when information can travel through them.
Thus, the depth scales that we identify provide bounds on how deep a network
may be trained for a specific choice of hyperparameters. As a corollary to
this, we argue that in networks at the edge of chaos, one of these depth scales
diverges. Thus arbitrarily deep networks may be trained only sufficiently close
to criticality. We show that the presence of dropout destroys the
order-to-chaos critical point and therefore strongly limits the maximum
trainable depth for random networks. Finally, we develop a mean field theory
for backpropagation and we show that the ordered and chaotic phases correspond
to regions of vanishing and exploding gradient respectively.
Pose2Instance: Harnessing Keypoints for Person Instance Segmentation
Authors: Subarna Tripathi, Maxwell Collins, Matthew Brown, Serge Belongie
Date: 2017-04-04 00:00:00
Tasks: Instance Segmentation, Segmentation, Semantic Segmentation

Human keypoints are a well-studied representation of people.We explore how to
use keypoint models to improve instance-level person segmentation. The main
idea is to harness the notion of a distance transform of oracle provided
keypoints or estimated keypoint heatmaps as a prior for person instance
segmentation task within a deep neural network. For training and evaluation, we
consider all those images from COCO where both instance segmentation and human
keypoints annotations are available. We first show how oracle keypoints can
boost the performance of existing human segmentation model during inference
without any training. Next, we propose a framework to directly learn a deep
instance segmentation model conditioned on human pose. Experimental results
show that at various Intersection Over Union (IOU) thresholds, in a constrained
environment with oracle keypoints, the instance segmentation accuracy achieves
10% to 12% relative improvements over a strong baseline of oracle bounding
boxes. In a more realistic environment, without the oracle keypoints, the
proposed deep person instance segmentation model conditioned on human pose
achieves 3.8% to 10.5% relative improvements comparing with its strongest
baseline of a deep network trained only for segmentation.