pwc-archive/papers-with-abstracts huggingface dataset:
Language-Based Image Editing with Recurrent Attentive Models
Authors: Jianbo Chen, Yelong Shen, Jianfeng Gao, Jingjing Liu, Xiaodong Liu
Date: 2017-11-16 00:00:00
Proceeding: CVPR 2018 6
Tasks: Colorization, Image Colorization, Image Segmentation, Segmentation, Semantic Segmentation

We investigate the problem of Language-Based Image Editing (LBIE). Given a
source image and a natural language description, we want to generate a target
image by editing the source image based on the description. We propose a
generic modeling framework for two sub-tasks of LBIE: language-based image
segmentation and image colorization. The framework uses recurrent attentive
models to fuse image and language features. Instead of using a fixed step size,
we introduce for each region of the image a termination gate to dynamically
determine after each inference step whether to continue extrapolating
additional information from the textual description. The effectiveness of the
framework is validated on three datasets. First, we introduce a synthetic
dataset, called CoSaL, to evaluate the end-to-end performance of our LBIE
system. Second, we show that the framework leads to state-of-the-art
performance on image segmentation on the ReferIt dataset. Third, we present the
first language-based colorization result on the Oxford-102 Flowers dataset.
EREL Selection using Morphological Relation
Authors: Yuying Li, Mehdi Faraji
Date: 2018-06-10 00:00:00
Tasks: Relation

This work concentrates on Extremal Regions of Extremum Level (EREL)
selection. EREL is a recently proposed feature detector aiming at detecting
regions from a set of extremal regions. This is a branching problem derived
from segmentation of arterial wall boundaries from Intravascular Ultrasound
(IVUS) images. For each IVUS frame, a set of EREL regions is generated to
describe the luminal area of human coronary. Each EREL is then fitted by an
ellipse to represent the luminal border. The goal is to assign the most
appropriate EREL as the lumen. In this work, EREL selection carries out in two
rounds. In the first round, the pattern in a set of EREL regions is analyzed
and used to generate an approximate luminal region. Then, the two-dimensional
(2D) correlation coefficients are computed between this approximate region and
each EREL to keep the ones with tightest relevance. In the second round, a
compactness measure is calculated for each EREL and its fitted ellipse to
guarantee that the resulting EREL has not affected by the common artifacts such
as bifurcations, shadows, and side branches. We evaluated the selected ERELs in
terms of Hausdorff Distance (HD) and Jaccard Measure (JM) on the train and test
set of a publicly available dataset. The results show that our selection
strategy outperforms the current state-of-the-art.
Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task
Authors: An Yang, Kai Liu, Jing Liu, Yajuan Lyu, Sujian Li
Date: 2018-06-10 00:00:00
Proceeding: WS 2018 7
Tasks: Machine Reading Comprehension, Question Answering, Reading Comprehension

Current evaluation metrics to question answering based machine reading
comprehension (MRC) systems generally focus on the lexical overlap between the
candidate and reference answers, such as ROUGE and BLEU. However, bias may
appear when these metrics are used for specific question types, especially
questions inquiring yes-no opinions and entity lists. In this paper, we make
adaptations on the metrics to better correlate n-gram overlap with the human
judgment for answers to these two question types. Statistical analysis proves
the effectiveness of our approach. Our adaptations may provide positive
guidance for the development of real-scene MRC systems.
Generative Adversarial Nets for Information Retrieval: Fundamentals and Advances
Authors: Wei-Nan Zhang
Date: 2018-06-10 00:00:00
Tasks: Information Retrieval, Retrieval, Text Generation

Generative adversarial nets (GANs) have been widely studied during the recent
development of deep learning and unsupervised learning. With an adversarial
training mechanism, GAN manages to train a generative model to fit the
underlying unknown real data distribution under the guidance of the
discriminative model estimating whether a data instance is real or generated.
Such a framework is originally proposed for fitting continuous data
distribution such as images, thus it is not straightforward to be directly
applied to information retrieval scenarios where the data is mostly discrete,
such as IDs, text and graphs. In this tutorial, we focus on discussing the GAN
techniques and the variants on discrete data fitting in various information
retrieval scenarios. (i) We introduce the fundamentals of GAN framework and its
theoretic properties; (ii) we carefully study the promising solutions to extend
GAN onto discrete data generation; (iii) we introduce IRGAN, the fundamental
GAN framework of fitting single ID data distribution and the direct application
on information retrieval; (iv) we further discuss the task of sequential
discrete data generation tasks, e.g., text generation, and the corresponding
GAN solutions; (v) we present the most recent work on graph/network data
fitting with node embedding techniques by GANs. Meanwhile, we also introduce
the relevant open-source platforms such as IRGAN and Texygen to help audience
conduct research experiments on GANs in information retrieval. Finally, we
conclude this tutorial with a comprehensive summarization and a prospect of
further research directions for GANs in information retrieval.
Instance Search via Instance Level Segmentation and Feature Representation
Authors: Yu Zhan, Wan-Lei Zhao
Date: 2018-06-10 00:00:00
Tasks: Instance Search, Instance Segmentation, Segmentation, Semantic Segmentation

Instance search is an interesting task as well as a challenging issue due to the lack of effective feature representation. In this paper, an instance level feature representation built upon fully convolutional instance-aware segmentation is proposed. The feature is ROI-pooled from the segmented instance region. So that instances in various sizes and layouts are represented by deep features in uniform length. This representation is further enhanced by the use of deformable ResNeXt blocks. Superior performance is observed in terms of its distinctiveness and scalability on a challenging evaluation dataset built by ourselves. In addition, the proposed enhancement on the network structure also shows superior performance on the instance segmentation task.
FMHash: Deep Hashing of In-Air-Handwriting for User Identification
Authors: Duo Lu, Dijiang Huang, Anshul Rai
Date: 2018-06-10 00:00:00
Tasks: Deep Hashing, User Identification

Many mobile systems and wearable devices, such as Virtual Reality (VR) or Augmented Reality (AR) headsets, lack a keyboard or touchscreen to type an ID and password for signing into a virtual website. However, they are usually equipped with gesture capture interfaces to allow the user to interact with the system directly with hand gestures. Although gesture-based authentication has been well-studied, less attention is paid to the gesture-based user identification problem, which is essentially an input method of account ID and an efficient searching and indexing method of a database of gesture signals. In this paper, we propose FMHash (i.e., Finger Motion Hash), a user identification framework that can generate a compact binary hash code from a piece of in-air-handwriting of an ID string. This hash code enables indexing and fast search of a large account database using the in-air-handwriting by a hash table. To demonstrate the effectiveness of the framework, we implemented a prototype and achieved >99.5% precision and >92.6% recall with exact hash code match on a dataset of 200 accounts collected by us. The ability of hashing in-air-handwriting pattern to binary code can be used to achieve convenient sign-in and sign-up with in-air-handwriting gesture ID on future mobile and wearable systems connected to the Internet.
Stationary Geometric Graphical Model Selection
Authors: Ilya Soloveychik, Vahid Tarokh
Date: 2018-06-10 00:00:00
Tasks: model, Model Selection, Time Series, Time Series Analysis

We consider the problem of model selection in Gaussian Markov fields in the
sample deficient scenario. In many practically important cases, the underlying
networks are embedded into Euclidean spaces. Using the natural geometric
structure, we introduce the notion of spatially stationary distributions over
geometric graphs. This directly generalizes the notion of stationary time
series to the multidimensional setting lacking time axis. We show that the idea
of spatial stationarity leads to a dramatic decrease in the sample complexity
of the model selection compared to abstract graphs with the same level of
sparsity. For geometric graphs on randomly spread vertices and edges of bounded
length, we develop tight information-theoretic bounds on sample complexity and
show that a finite number of independent samples is sufficient for a consistent
recovery. Finally, we develop an efficient technique capable of reliably and
consistently reconstructing graphs with a bounded number of measurements.
Neural Factor Graph Models for Cross-lingual Morphological Tagging
Authors: Chaitanya Malaviya, Matthew R. Gormley, Graham Neubig
Date: 2018-05-11 00:00:00
Proceeding: ACL 2018 7
Tasks: Morphological Analysis, Morphological Tagging, POS, TAG

Morphological analysis involves predicting the syntactic traits of a word
(e.g. {POS: Noun, Case: Acc, Gender: Fem}). Previous work in morphological
tagging improves performance for low-resource languages (LRLs) through
cross-lingual training with a high-resource language (HRL) from the same
family, but is limited by the strict, often false, assumption that tag sets
exactly overlap between the HRL and LRL. In this paper we propose a method for
cross-lingual morphological tagging that aims to improve information sharing
between languages by relaxing this assumption. The proposed model uses
factorial conditional random fields with neural network potentials, making it
possible to (1) utilize the expressive power of neural network representations
to smooth over superficial differences in the surface forms, (2) model pairwise
and transitive relationships between tags, and (3) accurately generate tag sets
that are unseen or rare in the training data. Experiments on four languages
from the Universal Dependencies Treebank demonstrate superior tagging
accuracies over existing cross-lingual approaches.
Explainable Recommendation via Multi-Task Learning in Opinionated Text Data
Authors: Nan Wang, Hongning Wang, Yiling Jia, Yue Yin
Date: 2018-06-10 00:00:00
Tasks: Explainable Recommendation, Multi-Task Learning

Explaining automatically generated recommendations allows users to make more
informed and accurate decisions about which results to utilize, and therefore
improves their satisfaction. In this work, we develop a multi-task learning
solution for explainable recommendation. Two companion learning tasks of user
preference modeling for recommendation} and \textit{opinionated content
modeling for explanation are integrated via a joint tensor factorization. As a
result, the algorithm predicts not only a user's preference over a list of
items, i.e., recommendation, but also how the user would appreciate a
particular item at the feature level, i.e., opinionated textual explanation.
Extensive experiments on two large collections of Amazon and Yelp reviews
confirmed the effectiveness of our solution in both recommendation and
explanation tasks, compared with several existing recommendation algorithms.
And our extensive user study clearly demonstrates the practical value of the
explainable recommendations generated by our algorithm.
Accurate Building Detection in VHR Remote Sensing Images using Geometric Saliency
Authors: Jin Huang, Gui-Song Xia, Fan Hu, Liangpei Zhang
Date: 2018-06-04 00:00:00

This paper aims to address the problem of detecting buildings from remote
sensing images with very high resolution (VHR). Inspired by the observation
that buildings are always more distinguishable in geometries than in texture or
spectral, we propose a new geometric building index (GBI) for accurate building
detection, which relies on the geometric saliency of building structures. The
geometric saliency of buildings is derived from a mid-level geometric
representations based on meaningful junctions that can locally describe
anisotropic geometrical structures of images. The resulting GBI is measured by
integrating the derived geometric saliency of buildings. Experiments on three
public datasets demonstrate that the proposed GBI achieves very promising
performance, and meanwhile shows impressive generalization capability.
RTSeg: Real-time Semantic Segmentation Comparative Study
Authors: Mennatullah Siam, Mostafa Gamal, Moemen Abdel-Razek, Senthil Yogamani, Martin Jagersand
Date: 2018-03-07 00:00:00
Tasks: Autonomous Driving, Benchmarking, Real-Time Semantic Segmentation, Segmentation, Semantic Segmentation

Semantic segmentation benefits robotics related applications especially autonomous driving. Most of the research on semantic segmentation is only on increasing the accuracy of segmentation models with little attention to computationally efficient solutions. The few work conducted in this direction does not provide principled methods to evaluate the different design choices for segmentation. In this paper, we address this gap by presenting a real-time semantic segmentation benchmarking framework with a decoupled design for feature extraction and decoding methods. The framework is comprised of different network architectures for feature extraction such as VGG16, Resnet18, MobileNet, and ShuffleNet. It is also comprised of multiple meta-architectures for segmentation that define the decoding methodology. These include SkipNet, UNet, and Dilation Frontend. Experimental results are presented on the Cityscapes dataset for urban scenes. The modular design allows novel architectures to emerge, that lead to 143x GFLOPs reduction in comparison to SegNet. This benchmarking framework is publicly available at "https://github.com/MSiam/TFSegmentation".
Generic Coreset for Scalable Learning of Monotonic Kernels: Logistic Regression, Sigmoid and more
Authors: Elad Tolochinsky, Ibrahim Jubran, Dan Feldman
Date: 2018-02-21 00:00:00
Tasks: regression

Coreset (or core-set) is a small weighted \emph{subset} $Q$ of an input set $P$ with respect to a given \emph{monotonic} function $f:\mathbb{R}\to\mathbb{R}$ that \emph{provably} approximates its fitting loss $\sum_{p\in P}f(p\cdot x)$ to \emph{any} given $x\in\mathbb{R}^d$. Using $Q$ we can obtain approximation of $x^*$ that minimizes this loss, by running \emph{existing} optimization algorithms on $Q$. In this work we provide: (i) A lower bound which proves that there are sets with no coresets smaller than $n=|P|$ for general monotonic loss functions. (ii) A proof that, under a natural assumption that holds e.g. for logistic regression and the sigmoid activation functions, a small coreset exists for \emph{any} input $P$. (iii) A generic coreset construction algorithm that computes such a small coreset $Q$ in $O(nd+n\log n)$ time, and (iv) Experimental results which demonstrate that our coresets are effective and are much smaller in practice than predicted in theory.
Building Bayesian Neural Networks with Blocks: On Structure, Interpretability and Uncertainty
Authors: Hao Henry Zhou, Yunyang Xiong, Vikas Singh
Date: 2018-06-10 00:00:00
Tasks: Gaussian Processes, Variational Inference

We provide simple schemes to build Bayesian Neural Networks (BNNs), block by
block, inspired by a recent idea of computation skeletons. We show how by
adjusting the types of blocks that are used within the computation skeleton, we
can identify interesting relationships with Deep Gaussian Processes (DGPs),
deep kernel learning (DKL), random features type approximation and other
topics. We give strategies to approximate the posterior via doubly stochastic
variational inference for such models which yield uncertainty estimates. We
give a detailed theoretical analysis and point out extensions that may be of
independent interest. As a special case, we instantiate our procedure to define
a Bayesian {\em additive} Neural network -- a promising strategy to identify
statistical interactions and has direct benefits for obtaining interpretable
models.
What Knowledge is Needed to Solve the RTE5 Textual Entailment Challenge?
Authors: Peter Clark
Date: 2018-06-10 00:00:00
Tasks: Natural Language Inference, RTE, World Knowledge

This document gives a knowledge-oriented analysis of about 20 interesting
Recognizing Textual Entailment (RTE) examples, drawn from the 2005 RTE5
competition test set. The analysis ignores shallow statistical matching
techniques between T and H, and rather asks: What would it take to reasonably
infer that T implies H? What world knowledge would be needed for this task?
Although such knowledge-intensive techniques have not had much success in RTE
evaluations, ultimately an intelligent system should be expected to know and
deploy this kind of world knowledge required to perform this kind of reasoning.
  The selected examples are typically ones which our RTE system (called BLUE)
got wrong and ones which require world knowledge to answer. In particular, the
analysis covers cases where there was near-perfect lexical overlap between T
and H, yet the entailment was NO, i.e., examples that most likely all current
RTE systems will have got wrong. A nice example is #341 (page 26), that
requires inferring from "a river floods" that "a river overflows its banks".
Seems it should be easy, right? Enjoy!
Semantic Correspondence: A Hierarchical Approach
Authors: Akila Pemasiri, Kien Nguyen, Sridha Sridhara, and Clinton Fookes
Date: 2018-06-10 00:00:00
Tasks: Semantic correspondence

Establishing semantic correspondence across images when the objects in the
images have undergone complex deformations remains a challenging task in the
field of computer vision. In this paper, we propose a hierarchical method to
tackle this problem by first semantically targeting the foreground objects to
localize the search space and then looking deeply into multiple levels of the
feature representation to search for point-level correspondence. In contrast to
existing approaches, which typically penalize large discrepancies, our approach
allows for significant displacements, with the aim to accommodate large
deformations of the objects in scene. Localizing the search space by
semantically matching object-level correspondence, our method robustly handles
large deformations of objects. Representing the target region by concatenated
hypercolumn features which take into account the hierarchical levels of the
surrounding context, helps to clear the ambiguity to further improve the
accuracy. By conducting multiple experiments across scenes with non-rigid
objects, we validate the proposed approach, and show that it outperforms the
state of the art methods for semantic correspondence establishment.
Genesis of Basic and Multi-Layer Echo State Network Recurrent Autoencoders for Efficient Data Representations
Authors: Naima Chouikhi, Boudour Ammar, Adel M. ALIMI
Date: 2018-04-24 00:00:00
Tasks: Classification, General Classification

It is a widely accepted fact that data representations intervene noticeably
in machine learning tools. The more they are well defined the better the
performance results are. Feature extraction-based methods such as autoencoders
are conceived for finding more accurate data representations from the original
ones. They efficiently perform on a specific task in terms of 1) high accuracy,
2) large short term memory and 3) low execution time. Echo State Network (ESN)
is a recent specific kind of Recurrent Neural Network which presents very rich
dynamics thanks to its reservoir-based hidden layer. It is widely used in
dealing with complex non-linear problems and it has outperformed classical
approaches in a number of tasks including regression, classification, etc. In
this paper, the noticeable dynamism and the large memory provided by ESN and
the strength of Autoencoders in feature extraction are gathered within an ESN
Recurrent Autoencoder (ESN-RAE). In order to bring up sturdier alternative to
conventional reservoir-based networks, not only single layer basic ESN is used
as an autoencoder, but also Multi-Layer ESN (ML-ESN-RAE). The new features,
once extracted from ESN's hidden layer, are applied to classification tasks.
The classification rates rise considerably compared to those obtained when
applying the original data features. An accuracy-based comparison is performed
between the proposed recurrent AEs and two variants of an ELM feed-forward AEs
(Basic and ML) in both of noise free and noisy environments. The empirical
study reveals the main contribution of recurrent connections in improving the
classification performance results.
Sparse Over-complete Patch Matching
Authors: Akila Pemasiri, Kien Nguyen, Sridha Sridharan, Clinton Fookes
Date: 2018-06-09 00:00:00
Tasks: Patch Matching

Image patch matching, which is the process of identifying corresponding
patches across images, has been used as a subroutine for many computer vision
and image processing tasks. State -of-the-art patch matching techniques take
image patches as input to a convolutional neural network to extract the patch
features and evaluate their similarity. Our aim in this paper is to improve on
the state of the art patch matching techniques by observing the fact that a
sparse-overcomplete representation of an image posses statistical properties of
natural visual scenes which can be exploited for patch matching. We propose a
new paradigm which encodes image patch details by encoding the patch and
subsequently using this sparse representation as input to a neural network to
compare the patches. As sparse coding is based on a generative model of natural
image patches, it can represent the patch in terms of the fundamental visual
components from which it has been composed of, leading to similar sparse codes
for patches which are built from similar components. Once the sparse coded
features are extracted, we employ a fully-connected neural network, which
captures the non-linear relationships between features, for comparison. We have
evaluated our approach using the Liberty and Notredame subsets of the popular
UBC patch dataset and set a new benchmark outperforming all state-of-the-art
patch matching techniques for these datasets.
Consistent Position Bias Estimation without Online Interventions for Learning-to-Rank
Authors: Aman Agarwal, Ivan Zaitsev, Thorsten Joachims
Date: 2018-06-09 00:00:00
Tasks: counterfactual, Learning-To-Rank, Position

Presentation bias is one of the key challenges when learning from implicit
feedback in search engines, as it confounds the relevance signal with
uninformative signals due to position in the ranking, saliency, and other
presentation factors. While it was recently shown how counterfactual
learning-to-rank (LTR) approaches \cite{Joachims/etal/17a} can provably
overcome presentation bias if observation propensities are known, it remains to
show how to accurately estimate these propensities. In this paper, we propose
the first method for producing consistent propensity estimates without manual
relevance judgments, disruptive interventions, or restrictive relevance
modeling assumptions. We merely require that we have implicit feedback data
from multiple different ranking functions. Furthermore, we argue that our
estimation technique applies to an extended class of Contextual Position-Based
Propensity Models, where propensities not only depend on position but also on
observable features of the query and document. Initial simulation studies
confirm that the approach is scalable, accurate, and robust.