pwc-archive/papers-with-abstracts huggingface dataset:
Learning to cluster in order to transfer across domains and tasks
Authors: Yen-Chang Hsu, Zhaoyang Lv, Zsolt Kira
Date: 2017-11-28 00:00:00
Proceeding: ICLR 2018 1
Tasks: Clustering, Constrained Clustering, Domain Adaptation, Transfer Learning, Unsupervised Domain Adaptation

This paper introduces a novel method to perform transfer learning across
domains and tasks, formulating it as a problem of learning to cluster. The key
insight is that, in addition to features, we can transfer similarity
information and this is sufficient to learn a similarity function and
clustering network to perform both domain adaptation and cross-task transfer
learning. We begin by reducing categorical information to pairwise constraints,
which only considers whether two instances belong to the same class or not.
This similarity is category-agnostic and can be learned from data in the source
domain using a similarity network. We then present two novel approaches for
performing transfer learning using this similarity function. First, for
unsupervised domain adaptation, we design a new loss function to regularize
classification with a constrained clustering loss, hence learning a clustering
network with the transferred similarity metric generating the training inputs.
Second, for cross-task learning (i.e., unsupervised clustering with unseen
categories), we propose a framework to reconstruct and estimate the number of
semantic clusters, again using the clustering network. Since the similarity
network is noisy, the key is to use a robust clustering algorithm, and we show
that our formulation is more robust than the alternative constrained and
unconstrained clustering approaches. Using this method, we first show state of
the art results for the challenging cross-task problem, applied on Omniglot and
ImageNet. Our results show that we can reconstruct semantic clusters with high
accuracy. We then evaluate the performance of cross-domain transfer using
images from the Office-31 and SVHN-MNIST tasks and present top accuracy on both
datasets. Our approach doesn't explicitly deal with domain discrepancy. If we
combine with a domain adaptation loss, it shows further improvement.
Mean Reverting Portfolios via Penalized OU-Likelihood Estimation
Authors: Jize Zhang, Tim Leung, Aleksandr Y. Aravkin
Date: 2018-03-17 00:00:00

We study an optimization-based approach to con- struct a mean-reverting
portfolio of assets. Our objectives are threefold: (1) design a portfolio that
is well-represented by an Ornstein-Uhlenbeck process with parameters estimated
by maximum likelihood, (2) select portfolios with desirable characteristics of
high mean reversion and low variance, and (3) select a parsimonious portfolio,
i.e. find a small subset of a larger universe of assets that can be used for
long and short positions. We present the full problem formulation, a
specialized algorithm that exploits partial minimization, and numerical
examples using both simulated and empirical price data.
Learning to Cluster for Proposal-Free Instance Segmentation
Authors: Yen-Chang Hsu, Zheng Xu, Zsolt Kira, Jiawei Huang
Date: 2018-03-17 00:00:00
Tasks: Autonomous Driving, Clustering, Instance Segmentation, Lane Detection, object-detection, Object Detection, Segmentation, Semantic Segmentation

This work proposed a novel learning objective to train a deep neural network
to perform end-to-end image pixel clustering. We applied the approach to
instance segmentation, which is at the intersection of image semantic
segmentation and object detection. We utilize the most fundamental property of
instance labeling -- the pairwise relationship between pixels -- as the
supervision to formulate the learning objective, then apply it to train a fully
convolutional network (FCN) for learning to perform pixel-wise clustering. The
resulting clusters can be used as the instance labeling directly. To support
labeling of an unlimited number of instance, we further formulate ideas from
graph coloring theory into the proposed learning objective. The evaluation on
the Cityscapes dataset demonstrates strong performance and therefore proof of
the concept. Moreover, our approach won the second place in the lane detection
competition of 2017 CVPR Autonomous Driving Challenge, and was the top
performer without using external data.
Experiments with Neural Networks for Small and Large Scale Authorship Verification
Authors: Marjan Hosseinia, Arjun Mukherjee
Date: 2018-03-17 00:00:00
Tasks: Articles, Authorship Verification

We propose two models for a special case of authorship verification problem.
The task is to investigate whether the two documents of a given pair are
written by the same author. We consider the authorship verification problem for
both small and large scale datasets. The underlying small-scale problem has two
main challenges: First, the authors of the documents are unknown to us because
no previous writing samples are available. Second, the two documents are short
(a few hundred to a few thousand words) and may differ considerably in the
genre and/or topic. To solve it we propose transformation encoder to transform
one document of the pair into the other. This document transformation generates
a loss which is used as a recognizable feature to verify if the authors of the
pair are identical. For the large scale problem where various authors are
engaged and more examples are available with larger length, a parallel
recurrent neural network is proposed. It compares the language models of the
two documents. We evaluate our methods on various types of datasets including
Authorship Identification datasets of PAN competition, Amazon reviews, and
machine learning articles. Experiments show that both methods achieve stable
and competitive performance compared to the baselines.
Constrained Deep Learning using Conditional Gradient and Applications in Computer Vision
Authors: Sathya N. Ravi, Tuan Dinh, Vishnu Sai Rao Lokhande, Vikas Singh
Date: 2018-03-17 00:00:00
Tasks: Image Inpainting

A number of results have recently demonstrated the benefits of incorporating
various constraints when training deep architectures in vision and machine
learning. The advantages range from guarantees for statistical generalization
to better accuracy to compression. But support for general constraints within
widely used libraries remains scarce and their broader deployment within many
applications that can benefit from them remains under-explored. Part of the
reason is that Stochastic gradient descent (SGD), the workhorse for training
deep neural networks, does not natively deal with constraints with global scope
very well. In this paper, we revisit a classical first order scheme from
numerical optimization, Conditional Gradients (CG), that has, thus far had
limited applicability in training deep models. We show via rigorous analysis
how various constraints can be naturally handled by modifications of this
algorithm. We provide convergence guarantees and show a suite of immediate
benefits that are possible -- from training ResNets with fewer layers but
better accuracy simply by substituting in our version of CG to faster training
of GANs with 50% fewer epochs in image inpainting applications to provably
better generalization guarantees using efficiently implementable forms of
recently proposed regularizers.
Note: Variational Encoding of Protein Dynamics Benefits from Maximizing Latent Autocorrelation
Authors: Hannah K. Wayment-Steele, Vijay S. Pande
Date: 2018-03-17 00:00:00

As deep Variational Auto-Encoder (VAE) frameworks become more widely used for
modeling biomolecular simulation data, we emphasize the capability of the VAE
architecture to concurrently maximize the timescale of the latent space while
inferring a reduced coordinate, which assists in finding slow processes as
according to the variational approach to conformational dynamics. We
additionally provide evidence that the VDE framework (Hern\'andez et al.,
2017), which uses this autocorrelation loss along with a time-lagged
reconstruction loss, obtains a variationally optimized latent coordinate in
comparison with related loss functions. We thus recommend leveraging the
autocorrelation of the latent space while training neural network models of
biomolecular simulation data to better represent slow processes.
Large-Scale Model Selection with Misspecification
Authors: Emre Demirkaya, Yang Feng, Pallavi Basu, Jinchi Lv
Date: 2018-03-17 00:00:00
Tasks: model, Model Selection

Model selection is crucial to high-dimensional learning and inference for
contemporary big data applications in pinpointing the best set of covariates
among a sequence of candidate interpretable models. Most existing work assumes
implicitly that the models are correctly specified or have fixed
dimensionality. Yet both features of model misspecification and high
dimensionality are prevalent in practice. In this paper, we exploit the
framework of model selection principles in misspecified models originated in Lv
and Liu (2014) and investigate the asymptotic expansion of Bayesian principle
of model selection in the setting of high-dimensional misspecified models. With
a natural choice of prior probabilities that encourages interpretability and
incorporates Kullback-Leibler divergence, we suggest the high-dimensional
generalized Bayesian information criterion with prior probability (HGBIC_p) for
large-scale model selection with misspecification. Our new information
criterion characterizes the impacts of both model misspecification and high
dimensionality on model selection. We further establish the consistency of
covariance contrast matrix estimation and the model selection consistency of
HGBIC_p in ultra-high dimensions under some mild regularity conditions. The
advantages of our new method are supported by numerical studies.
A Simple Method to improve Initialization Robustness for Active Contours driven by Local Region Fitting Energy
Authors: Keyan Ding, Linfang Xiao
Date: 2018-02-28 00:00:00

Active contour models based on local region fitting energy can segment images
with intensity inhomogeneity effectively, but their segmentation results are
easy to error if the initial contour is inappropriate. In this paper, we
present a simple and universal method of improving the robustness of initial
contour for these local fitting-based models. The core idea of proposed method
is exchanging the fitting values on the two sides of contour, so that the
fitting values inside the contour are always larger (or smaller) than the
values outside the contour in the process of curve evolution. In this way, the
whole curve will evolve along the inner (or outer) boundaries of object, and
less likely to be stuck in the object or background. Experimental results have
proved that using the proposed method can enhance the robustness of initial
contour and meanwhile keep the original advantages in the local fitting-based
models.
Datalog: Bag Semantics via Set Semantics
Authors: Leopoldo Bertossi, Georg Gottlob, Reinhard Pichler
Date: 2018-03-17 00:00:00
Tasks: Management, Translation

Duplicates in data management are common and problematic. In this work, we
present a translation of Datalog under bag semantics into a well-behaved
extension of Datalog, the so-called {\em warded Datalog}$^\pm$, under set
semantics. From a theoretical point of view, this allows us to reason on bag
semantics by making use of the well-established theoretical foundations of set
semantics. From a practical point of view, this allows us to handle the bag
semantics of Datalog by powerful, existing query engines for the required
extension of Datalog. This use of Datalog$^\pm$ is extended to give a set
semantics to duplicates in Datalog$^\pm$ itself. We investigate the properties
of the resulting Datalog$^\pm$ programs, the problem of deciding
multiplicities, and expressibility of some bag operations. Moreover, the
proposed translation has the potential for interesting applications such as to
Multiset Relational Algebra and the semantic web query language SPARQL with bag
semantics.
Quadruplet Network with One-Shot Learning for Fast Visual Object Tracking
Authors: Xingping Dong, Jianbing Shen, Dongming Wu, Kan Guo, Xiaogang Jin, Fatih Porikli
Date: 2017-05-19 00:00:00
Tasks: Object Tracking, One-Shot Learning, Triplet, Visual Object Tracking

In the same vein of discriminative one-shot learning, Siamese networks allow recognizing an object from a single exemplar with the same class label. However, they do not take advantage of the underlying structure of the data and the relationship among the multitude of samples as they only rely on pairs of instances for training. In this paper, we propose a new quadruplet deep network to examine the potential connections among the training instances, aiming to achieve a more powerful representation. We design four shared networks that receive multi-tuple of instances as inputs and are connected by a novel loss function consisting of pair-loss and triplet-loss. According to the similarity metric, we select the most similar and the most dissimilar instances as the positive and negative inputs of triplet loss from each multi-tuple. We show that this scheme improves the training performance. Furthermore, we introduce a new weight layer to automatically select suitable combination weights, which will avoid the conflict between triplet and pair loss leading to worse performance. We evaluate our quadruplet framework by model-free tracking-by-detection of objects from a single initial exemplar in several Visual Object Tracking benchmarks. Our extensive experimental analysis demonstrates that our tracker achieves superior performance with a real-time processing speed of 78 frames-per-second (fps).
A Novel Blaschke Unwinding Adaptive Fourier Decomposition based Signal Compression Algorithm with Application on ECG Signals
Authors: Chunyu Tan, Liming Zhang, Hau-Tieng Wu
Date: 2018-03-17 00:00:00
Tasks: Heart Rate Variability

This paper presents a novel signal compression algorithm based on the
Blaschke unwinding adaptive Fourier decomposition (AFD). The Blaschke unwinding
AFD is a newly developed signal decomposition theory. It utilizes the
Nevanlinna factorization and the maximal selection principle in each
decomposition step, and achieves a faster convergence rate with higher
fidelity. The proposed compression algorithm is applied to the
electrocardiogram signal. To assess the performance of the proposed compression
algorithm, in addition to the generic assessment criteria, we consider the less
discussed criteria related to the clinical needs -- for the heart rate
variability analysis purpose, how accurate the R peak information is preserved
is evaluated. The experiments are conducted on the MIT-BIH arrhythmia benchmark
database. The results show that the proposed algorithm performs better than
other state-of-the-art approaches. Meanwhile, it also well preserves the R peak
information.
A New Result on the Complexity of Heuristic Estimates for the A* Algorithm
Authors: Othar Hansson, Andrew Mayer, Marco Valtorta
Date: 2018-03-16 00:00:00

Relaxed models are abstract problem descriptions generated by ignoring
constraints that are present in base-level problems. They play an important
role in planning and search algorithms, as it has been shown that the length of
an optimal solution to a relaxed model yields a monotone heuristic for an A?
search of a base-level problem. Optimal solutions to a relaxed model may be
computed algorithmically or by search in a further relaxed model, leading to a
search that explores a hierarchy of relaxed models. In this paper, we review
the traditional definition of problem relaxation and show that searching in the
abstraction hierarchy created by problem relaxation will not reduce the
computational effort required to find optimal solutions to the base- level
problem, unless the relaxed problem found in the hierarchy can be transformed
by some optimization (e.g., subproblem factoring). Specifically, we prove that
any A* search of the base-level using a heuristic h2 will largely dominate an
A* search of the base-level using a heuristic h1, if h1 must be computed by an
A* search of the relaxed model using h2.
Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation
Authors: Heyi Li, Yunke Tian, Klaus Mueller, Xin Chen
Date: 2017-12-22 00:00:00
Tasks: Saliency Prediction

Despite the tremendous achievements of deep convolutional neural networks
(CNNs) in many computer vision tasks, understanding how they actually work
remains a significant challenge. In this paper, we propose a novel two-step
understanding method, namely Salient Relevance (SR) map, which aims to shed
light on how deep CNNs recognize images and learn features from areas, referred
to as attention areas, therein. Our proposed method starts out with a
layer-wise relevance propagation (LRP) step which estimates a pixel-wise
relevance map over the input image. Following, we construct a context-aware
saliency map, SR map, from the LRP-generated map which predicts areas close to
the foci of attention instead of isolated pixels that LRP reveals. In human
visual system, information of regions is more important than of pixels in
recognition. Consequently, our proposed approach closely simulates human
recognition. Experimental results using the ILSVRC2012 validation dataset in
conjunction with two well-established deep CNN models, AlexNet and VGG-16,
clearly demonstrate that our proposed approach concisely identifies not only
key pixels but also attention areas that contribute to the underlying neural
network's comprehension of the given images. As such, our proposed SR map
constitutes a convenient visual interface which unveils the visual attention of
the network and reveals which type of objects the model has learned to
recognize after training. The source code is available at
https://github.com/Hey1Li/Salient-Relevance-Propagation.
Learning to Segment via Cut-and-Paste
Authors: Tal Remez, Jonathan Huang, Matthew Brown
Date: 2018-03-16 00:00:00
Proceeding: ECCV 2018 9
Tasks: Instance Segmentation, Object, Segmentation, Semantic Segmentation

This paper presents a weakly-supervised approach to object instance
segmentation. Starting with known or predicted object bounding boxes, we learn
object masks by playing a game of cut-and-paste in an adversarial learning
setup. A mask generator takes a detection box and Faster R-CNN features, and
constructs a segmentation mask that is used to cut-and-paste the object into a
new image location. The discriminator tries to distinguish between real
objects, and those cut and pasted via the generator, giving a learning signal
that leads to improved object masks. We verify our method experimentally using
Cityscapes, COCO, and aerial image datasets, learning to segment objects
without ever having seen a mask in training. Our method exceeds the performance
of existing weakly supervised methods, without requiring hand-tuned segment
proposals, and reaches 90% of supervised performance.
Deep Component Analysis via Alternating Direction Neural Networks
Authors: Calvin Murdock, Ming-Fang Chang, Simon Lucey
Date: 2018-03-16 00:00:00
Proceeding: ECCV 2018 9
Tasks: Depth Estimation, Depth Prediction, Representation Learning

Despite a lack of theoretical understanding, deep neural networks have
achieved unparalleled performance in a wide range of applications. On the other
hand, shallow representation learning with component analysis is associated
with rich intuition and theory, but smaller capacity often limits its
usefulness. To bridge this gap, we introduce Deep Component Analysis (DeepCA),
an expressive multilayer model formulation that enforces hierarchical structure
through constraints on latent variables in each layer. For inference, we
propose a differentiable optimization algorithm implemented using recurrent
Alternating Direction Neural Networks (ADNNs) that enable parameter learning
using standard backpropagation. By interpreting feed-forward networks as
single-iteration approximations of inference in our model, we provide both a
novel theoretical perspective for understanding them and a practical technique
for constraining predictions with prior knowledge. Experimentally, we
demonstrate performance improvements on a variety of tasks, including
single-image depth prediction with sparse output constraints.
Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
Authors: Arun Mallya, Dillon Davis, Svetlana Lazebnik
Date: 2018-01-19 00:00:00
Proceeding: ECCV 2018 9
Tasks: Continual Learning, Quantization

This work presents a method for adapting a single, fixed deep neural network
to multiple tasks without affecting performance on already learned tasks. By
building upon ideas from network quantization and pruning, we learn binary
masks that piggyback on an existing network, or are applied to unmodified
weights of that network to provide good performance on a new task. These masks
are learned in an end-to-end differentiable fashion, and incur a low overhead
of 1 bit per network parameter, per task. Even though the underlying network is
fixed, the ability to mask individual weights allows for the learning of a
large number of filters. We show performance comparable to dedicated fine-tuned
networks for a variety of classification tasks, including those with large
domain shifts from the initial task (ImageNet), and a variety of network
architectures. Unlike prior work, we do not suffer from catastrophic forgetting
or competition between tasks, and our performance is agnostic to task ordering.
Code available at https://github.com/arunmallya/piggyback.
Evaluating Conditional Cash Transfer Policies with Machine Learning Methods
Authors: Tzai-Shuen Chen
Date: 2018-03-16 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

This paper presents an out-of-sample prediction comparison between major
machine learning models and the structural econometric model. Over the past
decade, machine learning has established itself as a powerful tool in many
prediction applications, but this approach is still not widely adopted in
empirical economic studies. To evaluate the benefits of this approach, I use
the most common machine learning algorithms, CART, C4.5, LASSO, random forest,
and adaboost, to construct prediction models for a cash transfer experiment
conducted by the Progresa program in Mexico, and I compare the prediction
results with those of a previous structural econometric study. Two prediction
tasks are performed in this paper: the out-of-sample forecast and the long-term
within-sample simulation. For the out-of-sample forecast, both the mean
absolute error and the root mean square error of the school attendance rates
found by all machine learning models are smaller than those found by the
structural model. Random forest and adaboost have the highest accuracy for the
individual outcomes of all subgroups. For the long-term within-sample
simulation, the structural model has better performance than do all of the
machine learning models. The poor within-sample fitness of the machine learning
model results from the inaccuracy of the income and pregnancy prediction
models. The result shows that the machine learning model performs better than
does the structural model when there are many data to learn; however, when the
data are limited, the structural model offers a more sensible prediction. The
findings of this paper show promise for adopting machine learning in economic
policy analyses in the era of big data.
Corpus Statistics in Text Classification of Online Data
Authors: Marina Sokolova, Victoria Bobicev
Date: 2018-03-16 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, Sentiment Analysis, text-classification, Text Classification

Transformation of Machine Learning (ML) from a boutique science to a
generally accepted technology has increased importance of reproduction and
transportability of ML studies. In the current work, we investigate how corpus
characteristics of textual data sets correspond to text classification results.
We work with two data sets gathered from sub-forums of an online health-related
forum. Our empirical results are obtained for a multi-class sentiment analysis
application.