pwc-archive/papers-with-abstracts huggingface dataset:
Smooth and Sparse Optimal Transport
Authors: Mathieu Blondel, Vivien Seguy, Antoine Rolet
Date: 2017-10-17 00:00:00

Entropic regularization is quickly emerging as a new standard in optimal
transport (OT). It enables to cast the OT computation as a differentiable and
unconstrained convex optimization problem, which can be efficiently solved
using the Sinkhorn algorithm. However, entropy keeps the transportation plan
strictly positive and therefore completely dense, unlike unregularized OT. This
lack of sparsity can be problematic in applications where the transportation
plan itself is of interest. In this paper, we explore regularizing the primal
and dual OT formulations with a strongly convex term, which corresponds to
relaxing the dual and primal constraints with smooth approximations. We show
how to incorporate squared $2$-norm and group lasso regularizations within that
framework, leading to sparse and group-sparse transportation plans. On the
theoretical side, we bound the approximation error introduced by regularizing
the primal and dual formulations. Our results suggest that, for the regularized
primal, the approximation error can often be smaller with squared $2$-norm than
with entropic regularization. We showcase our proposed framework on the task of
color transfer.
Provably Minimally-Distorted Adversarial Examples
Authors: Nicholas Carlini, Guy Katz, Clark Barrett, David L. Dill
Date: 2017-09-29 00:00:00

The ability to deploy neural networks in real-world, safety-critical systems
is severely limited by the presence of adversarial examples: slightly perturbed
inputs that are misclassified by the network. In recent years, several
techniques have been proposed for increasing robustness to adversarial examples
--- and yet most of these have been quickly shown to be vulnerable to future
attacks. For example, over half of the defenses proposed by papers accepted at
ICLR 2018 have already been broken. We propose to address this difficulty
through formal verification techniques. We show how to construct provably
minimally distorted adversarial examples: given an arbitrary neural network and
input sample, we can construct adversarial examples which we prove are of
minimal distortion. Using this approach, we demonstrate that one of the recent
ICLR defense proposals, adversarial retraining, provably succeeds at increasing
the distortion required to construct adversarial examples by a factor of 4.2.
A survey on trajectory clustering analysis
Authors: Jiang Bian, Dayong Tian, Yuanyan Tang, DaCheng Tao
Date: 2018-02-20 00:00:00
Tasks: Clustering, Survey, Trajectory Clustering

This paper comprehensively surveys the development of trajectory clustering.
Considering the critical role of trajectory data mining in modern intelligent
systems for surveillance security, abnormal behavior detection, crowd behavior
analysis, and traffic control, trajectory clustering has attracted growing
attention. Existing trajectory clustering methods can be grouped into three
categories: unsupervised, supervised and semi-supervised algorithms. In spite
of achieving a certain level of development, trajectory clustering is limited
in its success by complex conditions such as application scenarios and data
dimensions. This paper provides a holistic understanding and deep insight into
trajectory clustering, and presents a comprehensive analysis of representative
methods and promising future directions.
Learning from Longitudinal Face Demonstration - Where Tractable Deep Modeling Meets Inverse Reinforcement Learning
Authors: Chi Nhan Duong, Kha Gia Quach, Khoa Luu, T. Hoang Ngan Le, Marios Savvides, Tien D. Bui
Date: 2017-11-28 00:00:00
Tasks: Face Verification, MORPH, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper presents a novel Subject-dependent Deep Aging Path (SDAP), which
inherits the merits of both Generative Probabilistic Modeling and Inverse
Reinforcement Learning to model the facial structures and the longitudinal face
aging process of a given subject. The proposed SDAP is optimized using
tractable log-likelihood objective functions with Convolutional Neural Networks
(CNNs) based deep feature extraction. Instead of applying a fixed aging
development path for all input faces and subjects, SDAP is able to provide the
most appropriate aging development path for individual subject that optimizes
the reward aging formulation. Unlike previous methods that can take only one
image as the input, SDAP further allows multiple images as inputs, i.e. all
information of a subject at either the same or different ages, to produce the
optimal aging path for the given subject. Finally, SDAP allows efficiently
synthesizing in-the-wild aging faces. The proposed model is experimented in
both tasks of face aging synthesis and cross-age face verification. The
experimental results consistently show SDAP achieves the state-of-the-art
performance on numerous face aging databases, i.e. FG-NET, MORPH, AginG Faces
in the Wild (AGFW), and Cross-Age Celebrity Dataset (CACD). Furthermore, we
also evaluate the performance of SDAP on large-scale Megaface challenge to
demonstrate the advantages of the proposed solution.
Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach
Authors: Ming Yu, Varun Gupta, Mladen Kolar
Date: 2018-02-20 00:00:00
Tasks: Multi-Task Learning

We study the problem of recovery of matrices that are simultaneously low rank
and row and/or column sparse. Such matrices appear in recent applications in
cognitive neuroscience, imaging, computer vision, macroeconomics, and genetics.
We propose a GDT (Gradient Descent with hard Thresholding) algorithm to
efficiently recover matrices with such structure, by minimizing a bi-convex
function over a nonconvex set of constraints. We show linear convergence of the
iterates obtained by GDT to a region within statistical error of an optimal
solution. As an application of our method, we consider multi-task learning
problems and show that the statistical error rate obtained by GDT is near
optimal compared to minimax rate. Experiments demonstrate competitive
performance and much faster running speed compared to existing methods, on both
simulations and real data sets.
A Unified Framework for Planning in Adversarial and Cooperative Environments
Authors: Anagha Kulkarni, Siddharth Srivastava, Subbarao Kambhampati
Date: 2018-02-16 00:00:00

Users of AI systems may rely upon them to produce plans for achieving desired
objectives. Such AI systems should be able to compute obfuscated plans whose
execution in adversarial situations protects privacy, as well as legible plans
which are easy for team members to understand in cooperative situations. We
develop a unified framework that addresses these dual problems by computing
plans with a desired level of comprehensibility from the point of view of a
partially informed observer. For adversarial settings, our approach produces
obfuscated plans with observations that are consistent with at least k goals
from a set of decoy goals. By slightly varying our framework, we present an
approach for goal legibility in cooperative settings which produces plans that
achieve a goal while being consistent with at most j goals from a set of
confounding goals. In addition, we show how the observability of the observer
can be controlled to either obfuscate or clarify the next actions in a plan
when the goal is known to the observer. We present theoretical results on the
complexity analysis of our problems. We demonstrate the execution of obfuscated
and legible plans in a cooking domain using a physical robot Fetch. We also
provide an empirical evaluation to show the feasibility and usefulness of our
approaches using IPC domains.
Co-occurrence matrix analysis-based semi-supervised training for object detection
Authors: Min-Kook Choi, Jaehyeong Park, Jihun Jung, Heechul Jung, Jin-Hee Lee, Woong Jae Won, Woo Young Jung, Jincheol Kim, Soon Kwon
Date: 2018-02-20 00:00:00
Tasks: Object, object-detection, Object Detection, Object Recognition, Semantic Segmentation

One of the most important factors in training object recognition networks
using convolutional neural networks (CNNs) is the provision of annotated data
accompanying human judgment. Particularly, in object detection or semantic
segmentation, the annotation process requires considerable human effort. In
this paper, we propose a semi-supervised learning (SSL)-based training
methodology for object detection, which makes use of automatic labeling of
un-annotated data by applying a network previously trained from an annotated
dataset. Because an inferred label by the trained network is dependent on the
learned parameters, it is often meaningless for re-training the network. To
transfer a valuable inferred label to the unlabeled data, we propose a
re-alignment method based on co-occurrence matrix analysis that takes into
account one-hot-vector encoding of the estimated label and the correlation
between the objects in the image. We used an MS-COCO detection dataset to
verify the performance of the proposed SSL method and deformable neural
networks (D-ConvNets) as an object detector for basic training. The performance
of the existing state-of-the-art detectors (DConvNets, YOLO v2, and single shot
multi-box detector (SSD)) can be improved by the proposed SSL method without
using the additional model parameter or modifying the network architecture.
Learning Pain from Action Unit Combinations: A Weakly Supervised Approach via Multiple Instance Learning
Authors: Zhanli Chen, Rashid Ansari, Diana J. Wilkie
Date: 2017-12-05 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER), Multiple Instance Learning, Weakly-supervised Learning

Patient pain can be detected highly reliably from facial expressions using a
set of facial muscle-based action units (AUs) defined by the Facial Action
Coding System (FACS). A key characteristic of facial expression of pain is the
simultaneous occurrence of pain-related AU combinations, whose automated
detection would be highly beneficial for efficient and practical pain
monitoring. Existing general Automated Facial Expression Recognition (AFER)
systems prove inadequate when applied specifically for detecting pain as they
either focus on detecting individual pain-related AUs but not on combinations
or they seek to bypass AU detection by training a binary pain classifier
directly on pain intensity data but are limited by lack of enough labeled data
for satisfactory training. In this paper, we propose a new approach that mimics
the strategy of human coders of decoupling pain detection into two consecutive
tasks: one performed at the individual video-frame level and the other at
video-sequence level. Using state-of-the-art AFER tools to detect single AUs at
the frame level, we propose two novel data structures to encode AU combinations
from single AU scores. Two weakly supervised learning frameworks namely
multiple instance learning (MIL) and multiple clustered instance learning
(MCIL) are employed corresponding to each data structure to learn pain from
video sequences. Experimental results show an 87% pain recognition accuracy
with 0.94 AUC (Area Under Curve) on the UNBC-McMaster Shoulder Pain Expression
dataset. Tests on long videos in a lung cancer patient video dataset
demonstrates the potential value of the proposed system for pain monitoring in
clinical settings.
Neural Network Ensembles to Real-time Identification of Plug-level Appliance Measurements
Authors: Karim Said Barsim, Lukas Mauch, Bin Yang
Date: 2018-02-20 00:00:00
Tasks: Non-Intrusive Load Monitoring

The problem of identifying end-use electrical appliances from their
individual consumption profiles, known as the appliance identification problem,
is a primary stage in both Non-Intrusive Load Monitoring (NILM) and automated
plug-wise metering. Therefore, appliance identification has received dedicated
studies with various electric appliance signatures, classification models, and
evaluation datasets. In this paper, we propose a neural network ensembles
approach to address this problem using high resolution measurements. The models
are trained on the raw current and voltage waveforms, and thus, eliminating the
need for well engineered appliance signatures. We evaluate the proposed model
on a publicly available appliance dataset from 55 residential buildings, 11
appliance categories, and over 1000 measurements. We further study the
stability of the trained models with respect to training dataset, sampling
frequency, and variations in the steady-state operation of appliances.
Agile Amulet: Real-Time Salient Object Detection with Contextual Attention
Authors: Pingping Zhang, Luyao Wang, Dong Wang, Huchuan Lu, Chunhua Shen
Date: 2018-02-20 00:00:00
Tasks: object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection

This paper proposes an Agile Aggregating Multi-Level feaTure framework (Agile
Amulet) for salient object detection. The Agile Amulet builds on previous works
to predict saliency maps using multi-level convolutional features. Compared to
previous works, Agile Amulet employs some key innovations to improve training
and testing speed while also increase prediction accuracy. More specifically,
we first introduce a contextual attention module that can rapidly highlight
most salient objects or regions with contextual pyramids. Thus, it effectively
guides the learning of low-layer convolutional features and tells the backbone
network where to look. The contextual attention module is a fully convolutional
mechanism that simultaneously learns complementary features and predicts
saliency scores at each pixel. In addition, we propose a novel method to
aggregate multi-level deep convolutional features. As a result, we are able to
use the integrated side-output features of pre-trained convolutional networks
alone, which significantly reduces the model parameters leading to a model size
of 67 MB, about half of Amulet. Compared to other deep learning based saliency
methods, Agile Amulet is of much lighter-weight, runs faster (30 fps in
real-time) and achieves higher performance on seven public benchmarks in terms
of both quantitative and qualitative evaluation.
Variance-based Gradient Compression for Efficient Distributed Deep Learning
Authors: Yusuke Tsuzuku, Hiroto Imachi, Takuya Akiba
Date: 2018-02-16 00:00:00
Proceeding: ICLR 2018 1
Tasks: Deep Learning

Due to the substantial computational cost, training state-of-the-art deep
neural networks for large-scale datasets often requires distributed training
using multiple computation workers. However, by nature, workers need to
frequently communicate gradients, causing severe bottlenecks, especially on
lower bandwidth connections. A few methods have been proposed to compress
gradient for efficient communication, but they either suffer a low compression
ratio or significantly harm the resulting model accuracy, particularly when
applied to convolutional neural networks. To address these issues, we propose a
method to reduce the communication overhead of distributed deep learning. Our
key observation is that gradient updates can be delayed until an unambiguous
(high amplitude, low variance) gradient has been calculated. We also present an
efficient algorithm to compute the variance with negligible additional cost. We
experimentally show that our method can achieve very high compression ratio
while maintaining the result model accuracy. We also analyze the efficiency
using computation and communication cost models and provide the evidence that
this method enables distributed deep learning for many scenarios with commodity
environments.
TAP-DLND 1.0 : A Corpus for Document Level Novelty Detection
Authors: Tirthankar Ghosal, Amitra Salam, Swati Tiwari, Asif Ekbal, Pushpak Bhattacharyya
Date: 2018-02-20 00:00:00
Proceeding: LREC 2018 5
Tasks: Articles, Benchmarking, Document Summarization, Extractive Document Summarization, Extractive Text Summarization, General Classification, Novelty Detection

Detecting novelty of an entire document is an Artificial Intelligence (AI)
frontier problem that has widespread NLP applications, such as extractive
document summarization, tracking development of news events, predicting impact
of scholarly articles, etc. Important though the problem is, we are unaware of
any benchmark document level data that correctly addresses the evaluation of
automatic novelty detection techniques in a classification framework. To bridge
this gap, we present here a resource for benchmarking the techniques for
document level novelty detection. We create the resource via event-specific
crawling of news documents across several domains in a periodic manner. We
release the annotated corpus with necessary statistics and show its use with a
developed system for the problem in concern.
Efficient Embedding of MPI Collectives in MXNET DAGs for scaling Deep Learning
Authors: Amith R Mamidala
Date: 2018-02-20 00:00:00

Availability of high performance computing infrastructures such as clusters
of GPUs and CPUs have fueled the growth of distributed learning systems. Deep
Learning frameworks express neural nets as DAGs and execute these DAGs on
computation resources such as GPUs. In this paper, we propose efficient designs
of embedding MPI collective operations into data parallel DAGs. Incorrect
designs can easily lead to deadlocks or program crashes. In particular, we
demonstrate three designs: Funneled, Concurrent communication and Dependency
chaining of using MPI collectives with DAGs. These designs automatically enable
overlap of computation with communication by allowing for concurrent execution
with the other tasks. We directly implement these designs into the KVStore API
of the MXNET. This allows us to directly leverage the rest of the
infrastructure. Using ImageNet and CIFAR data sets, we show the potential of
our designs. In particular, our designs scale to 256 GPUs with as low as 50
seconds of epoch times for ImageNet 1K datasets.
Fast and robust misalignment correction of Fourier ptychographic microscopy
Authors: Ao Zhou, Wei Wang, Ni Chen, Edmund Y. Lam, Byoungho Lee, Guohai Situ
Date: 2018-02-20 00:00:00
Tasks: Sensitivity

Fourier ptychographi cmicroscopy(FPM) is a newly developed computational
imaging technique that can provide gigapixel images with both high resolution
(HR) and wide field of view (FOV). However, the positional misalignment of the
LED array induces a degradation of the reconstruction, especially in the
regions away from the optical axis. In this paper, we propose a robust and fast
method to correct the LED misalignment of FPM, termed as misalignment
correction for FPM (mcFPM). Although different regions in the FOV have
different sensitivity to the LED misalignment, the experimental results show
that mcFPM is robust to eliminate the degradation in each region. Compared with
the state-of-the-art methods, mcFPM is much faster.
EA-CG: An Approximate Second-Order Method for Training Fully-Connected Neural Networks
Authors: Sheng-Wei Chen, Chun-Nan Chou, Edward Y. Chang
Date: 2018-02-19 00:00:00

For training fully-connected neural networks (FCNNs), we propose a practical
approximate second-order method including: 1) an approximation of the Hessian
matrix and 2) a conjugate gradient (CG) based method. Our proposed approximate
Hessian matrix is memory-efficient and can be applied to any FCNNs where the
activation and criterion functions are twice differentiable. We devise a
CG-based method incorporating one-rank approximation to derive Newton
directions for training FCNNs, which significantly reduces both space and time
complexity. This CG-based method can be employed to solve any linear equation
where the coefficient matrix is Kronecker-factored, symmetric and positive
definite. Empirical studies show the efficacy and efficiency of our proposed
method.
DeepThin: A Self-Compressing Library for Deep Neural Networks
Authors: Matthew Sotoudeh, Sara S. Baghsorkhi
Date: 2018-02-20 00:00:00

As the industry deploys increasingly large and complex neural networks to
mobile devices, more pressure is put on the memory and compute resources of
those devices. Deep compression, or compression of deep neural network weight
matrices, is a technique to stretch resources for such scenarios. Existing
compression methods cannot effectively compress models smaller than 1-2% of
their original size. We develop a new compression technique, DeepThin, building
on existing research in the area of low rank factorization. We identify and
break artificial constraints imposed by low rank approximations by combining
rank factorization with a reshaping process that adds nonlinearity to the
approximation function. We deploy DeepThin as a plug-gable library integrated
with TensorFlow that enables users to seamlessly compress models at different
granularities. We evaluate DeepThin on two state-of-the-art acoustic models,
TFKaldi and DeepSpeech, comparing it to previous compression work (Pruning,
HashNet, and Rank Factorization), empirical limit study approaches, and
hand-tuned models. For TFKaldi, our DeepThin networks show better word error
rates (WER) than competing methods at practically all tested compression rates,
achieving an average of 60% relative improvement over rank factorization, 57%
over pruning, 23% over hand-tuned same-size networks, and 6% over the
computationally expensive HashedNets. For DeepSpeech, DeepThin-compressed
networks achieve better test loss than all other compression methods, reaching
a 28% better result than rank factorization, 27% better than pruning, 20%
better than hand-tuned same-size networks, and 12% better than HashedNets.
DeepThin also provide inference performance benefits ranging from 2X to 14X
speedups, depending on the compression ratio and platform cache sizes.
Comparison Based Learning from Weak Oracles
Authors: Ehsan Kazemi, Lin Chen, Sanjoy Dasgupta, Amin Karbasi
Date: 2018-02-20 00:00:00

There is increasing interest in learning algorithms that involve interaction
between human and machine. Comparison-based queries are among the most natural
ways to get feedback from humans. A challenge in designing comparison-based
interactive learning algorithms is coping with noisy answers. The most common
fix is to submit a query several times, but this is not applicable in many
situations due to its prohibitive cost and due to the unrealistic assumption of
independent noise in different repetitions of the same query.
  In this paper, we introduce a new weak oracle model, where a non-malicious
user responds to a pairwise comparison query only when she is quite sure about
the answer. This model is able to mimic the behavior of a human in noise-prone
regions. We also consider the application of this weak oracle model to the
problem of content search (a variant of the nearest neighbor search problem)
through comparisons. More specifically, we aim at devising efficient algorithms
to locate a target object in a database equipped with a dissimilarity metric
via invocation of the weak comparison oracle. We propose two algorithms termed
WORCS-I and WORCS-II (Weak-Oracle Comparison-based Search), which provably
locate the target object in a number of comparisons close to the entropy of the
target distribution. While WORCS-I provides better theoretical guarantees,
WORCS-II is applicable to more technically challenging scenarios where the
algorithm has limited access to the ranking dissimilarity between objects. A
series of experiments validate the performance of our proposed algorithms.