pwc-archive/papers-with-abstracts huggingface dataset:
Boosting the accuracy of multi-spectral image pan-sharpening by learning a deep residual network
Authors: Yancong Wei, Qiangqiang Yuan, Huanfeng Shen, Liangpei Zhang
Date: 2017-05-22 00:00:00

In the field of fusing multi-spectral and panchromatic images
(Pan-sharpening), the impressive effectiveness of deep neural networks has been
recently employed to overcome the drawbacks of traditional linear models and
boost the fusing accuracy. However, to the best of our knowledge, existing
research works are mainly based on simple and flat networks with relatively
shallow architecture, which severely limited their performances. In this paper,
the concept of residual learning has been introduced to form a very deep
convolutional neural network to make a full use of the high non-linearity of
deep learning models. By both quantitative and visual assessments on a large
number of high quality multi-spectral images from various sources, it has been
supported that our proposed model is superior to all mainstream algorithms
included in the comparison, and achieved the highest spatial-spectral unified
accuracy.
Logical Learning Through a Hybrid Neural Network with Auxiliary Inputs
Authors: Fang Wan, Chaoyang Song
Date: 2017-05-23 00:00:00

The human reasoning process is seldom a one-way process from an input leading
to an output. Instead, it often involves a systematic deduction by ruling out
other possible outcomes as a self-checking mechanism. In this paper, we
describe the design of a hybrid neural network for logical learning that is
similar to the human reasoning through the introduction of an auxiliary input,
namely the indicators, that act as the hints to suggest logical outcomes. We
generate these indicators by digging into the hidden information buried
underneath the original training data for direct or indirect suggestions. We
used the MNIST data to demonstrate the design and use of these indicators in a
convolutional neural network. We trained a series of such hybrid neural
networks with variations of the indicators. Our results show that these hybrid
neural networks are very robust in generating logical outcomes with inherently
higher prediction accuracy than the direct use of the original input and output
in apparent models. Such improved predictability with reassured logical
confidence is obtained through the exhaustion of all possible indicators to
rule out all illogical outcomes, which is not available in the apparent models.
Our logical learning process can effectively cope with the unknown unknowns
using a full exploitation of all existing knowledge available for learning. The
design and implementation of the hints, namely the indicators, become an
essential part of artificial intelligence for logical learning. We also
introduce an ongoing application setup for this hybrid neural network in an
autonomous grasping robot, namely as_DeepClaw, aiming at learning an optimized
grasping pose through logical learning.
Learning to Succeed while Teaching to Fail: Privacy in Closed Machine Learning Systems
Authors: Jure Sokolic, Qiang Qiu, Miguel R. D. Rodrigues, Guillermo Sapiro
Date: 2017-05-23 00:00:00
Tasks: BIG-bench Machine Learning, Blocking, Fairness

Security, privacy, and fairness have become critical in the era of data
science and machine learning. More and more we see that achieving universally
secure, private, and fair systems is practically impossible. We have seen for
example how generative adversarial networks can be used to learn about the
expected private training data; how the exploitation of additional data can
reveal private information in the original one; and how what looks like
unrelated features can teach us about each other. Confronted with this
challenge, in this paper we open a new line of research, where the security,
privacy, and fairness is learned and used in a closed environment. The goal is
to ensure that a given entity (e.g., the company or the government), trusted to
infer certain information with our data, is blocked from inferring protected
information from it. For example, a hospital might be allowed to produce
diagnosis on the patient (the positive task), without being able to infer the
gender of the subject (negative task). Similarly, a company can guarantee that
internally it is not using the provided data for any undesired task, an
important goal that is not contradicting the virtually impossible challenge of
blocking everybody from the undesired task. We design a system that learns to
succeed on the positive task while simultaneously fail at the negative one, and
illustrate this with challenging cases where the positive task is actually
harder than the negative one being blocked. Fairness, to the information in the
negative task, is often automatically obtained as a result of this proposed
approach. The particular framework and examples open the door to security,
privacy, and fairness in very important closed scenarios, ranging from private
data accumulation companies like social networks to law-enforcement and
hospitals.
Robust Large Margin Deep Neural Networks
Authors: Jure Sokolic, Raja Giryes, Guillermo Sapiro, Miguel R. D. Rodrigues
Date: 2016-05-26 00:00:00

The generalization error of deep neural networks via their classification
margin is studied in this work. Our approach is based on the Jacobian matrix of
a deep neural network and can be applied to networks with arbitrary
non-linearities and pooling layers, and to networks with different
architectures such as feed forward networks and residual networks. Our analysis
leads to the conclusion that a bounded spectral norm of the network's Jacobian
matrix in the neighbourhood of the training samples is crucial for a deep
neural network of arbitrary depth and width to generalize well. This is a
significant improvement over the current bounds in the literature, which imply
that the generalization error grows with either the width or the depth of the
network. Moreover, it shows that the recently proposed batch normalization and
weight normalization re-parametrizations enjoy good generalization properties,
and leads to a novel network regularizer based on the network's Jacobian
matrix. The analysis is supported with experimental results on the MNIST,
CIFAR-10, LaRED and ImageNet datasets.
Unbiased Online Recurrent Optimization
Authors: Corentin Tallec, Yann Ollivier
Date: 2017-02-16 00:00:00
Proceeding: ICLR 2018 1

The novel Unbiased Online Recurrent Optimization (UORO) algorithm allows for
online learning of general recurrent computational graphs such as recurrent
network models. It works in a streaming fashion and avoids backtracking through
past activations and inputs. UORO is computationally as costly as Truncated
Backpropagation Through Time (truncated BPTT), a widespread algorithm for
online learning of recurrent networks. UORO is a modification of NoBackTrack
that bypasses the need for model sparsity and makes implementation easy in
current deep learning frameworks, even for complex models.
  Like NoBackTrack, UORO provides unbiased gradient estimates; unbiasedness is
the core hypothesis in stochastic gradient descent theory, without which
convergence to a local optimum is not guaranteed. On the contrary, truncated
BPTT does not provide this property, leading to possible divergence.
  On synthetic tasks where truncated BPTT is shown to diverge, UORO converges.
For instance, when a parameter has a positive short-term but negative long-term
influence, truncated BPTT diverges unless the truncation span is very
significantly longer than the intrinsic temporal range of the interactions,
while UORO performs well thanks to the unbiasedness of its gradients.
Iterative Bayesian Learning for Crowdsourced Regression
Authors: Jungseul Ok, Sewoong Oh, Yunhun Jang, Jinwoo Shin, Yung Yi
Date: 2017-02-28 00:00:00
Tasks: regression

Crowdsourcing platforms emerged as popular venues for purchasing human
intelligence at low cost for large volume of tasks. As many low-paid workers
are prone to give noisy answers, a common practice is to add redundancy by
assigning multiple workers to each task and then simply average out these
answers. However, to fully harness the wisdom of the crowd, one needs to learn
the heterogeneous quality of each worker. We resolve this fundamental challenge
in crowdsourced regression tasks, i.e., the answer takes continuous labels,
where identifying good or bad workers becomes much more non-trivial compared to
a classification setting of discrete labels. In particular, we introduce a
Bayesian iterative scheme and show that it provably achieves the optimal mean
squared error. Our evaluations on synthetic and real-world datasets support our
theoretical results and show the superiority of the proposed scheme.
Correlation Alignment by Riemannian Metric for Domain Adaptation
Authors: Pietro Morerio, Vittorio Murino
Date: 2017-05-23 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Domain adaptation techniques address the problem of reducing the sensitivity
of machine learning methods to the so-called domain shift, namely the
difference between source (training) and target (test) data distributions. In
particular, unsupervised domain adaptation assumes no labels are available in
the target domain. To this end, aligning second order statistics (covariances)
of target and source domains have proven to be an effective approach ti fill
the gap between the domains. However, covariance matrices do not form a
subspace of the Euclidean space, but live in a Riemannian manifold with
non-positive curvature, making the usual Euclidean metric suboptimal to measure
distances. In this paper, we extend the idea of training a neural network with
a constraint on the covariances of the hidden layer features, by rigorously
accounting for the curved structure of the manifold of symmetric positive
definite matrices. The resulting loss function exploits a theoretically sound
geodesic distance on such manifold. Results show indeed the suboptimal nature
of the Euclidean distance. This makes us able to perform better than previous
approaches on the standard Office dataset, a benchmark for domain adaptation
techniques.
Weakly Supervised PLDA Training
Authors: Lantian Li, Yixiang Chen, Dong Wang, Chenghui Zhao
Date: 2016-09-27 00:00:00
Tasks: Speaker Verification

PLDA is a popular normalization approach for the i-vector model, and it has
delivered state-of-the-art performance in speaker verification. However, PLDA
training requires a large amount of labelled development data, which is highly
expensive in most cases. We present a cheap PLDA training approach, which
assumes that speakers in the same session can be easily separated, and speakers
in different sessions are simply different. This results in `weak labels' which
are not fully accurate but cheap, leading to a weak PLDA training.
  Our experimental results on real-life large-scale telephony customer service
achieves demonstrated that the weak training can offer good performance when
human-labelled data are limited. More interestingly, the weak training can be
employed as a discriminative adaptation approach, which is more efficient than
the prevailing unsupervised method when human-labelled data are insufficient.
Collaborative Learning for Language and Speaker Recognition
Authors: Lantian Li, Zhiyuan Tang, Dong Wang, Andrew Abel, Yang Feng, Shiyue Zhang
Date: 2016-09-27 00:00:00
Tasks: Speaker Recognition

This paper presents a unified model to perform language and speaker
recognition simultaneously and altogether. The model is based on a multi-task
recurrent neural network where the output of one task is fed as the input of
the other, leading to a collaborative learning framework that can improve both
language and speaker recognition by borrowing information from each other. Our
experiments demonstrated that the multi-task model outperforms the
task-specific models on both tasks.
Black-Box Attacks against RNN based Malware Detection Algorithms
Authors: Weiwei Hu, Ying Tan
Date: 2017-05-23 00:00:00
Tasks: Malware Detection

Recent researches have shown that machine learning based malware detection
algorithms are very vulnerable under the attacks of adversarial examples. These
works mainly focused on the detection algorithms which use features with fixed
dimension, while some researchers have begun to use recurrent neural networks
(RNN) to detect malware based on sequential API features. This paper proposes a
novel algorithm to generate sequential adversarial examples, which are used to
attack a RNN based malware detection system. It is usually hard for malicious
attackers to know the exact structures and weights of the victim RNN. A
substitute RNN is trained to approximate the victim RNN. Then we propose a
generative RNN to output sequential adversarial examples from the original
sequential malware inputs. Experimental results showed that RNN based malware
detection algorithms fail to detect most of the generated malicious adversarial
examples, which means the proposed model is able to effectively bypass the
detection algorithms.
SIFT Meets CNN: A Decade Survey of Instance Retrieval
Authors: Liang Zheng, Yi Yang, Qi Tian
Date: 2016-08-05 00:00:00
Tasks: Content-Based Image Retrieval, Image Retrieval, Retrieval, Survey

In the early days, content-based image retrieval (CBIR) was studied with
global features. Since 2003, image retrieval based on local descriptors (de
facto SIFT) has been extensively studied for over a decade due to the advantage
of SIFT in dealing with image transformations. Recently, image representations
based on the convolutional neural network (CNN) have attracted increasing
interest in the community and demonstrated impressive performance. Given this
time of rapid evolution, this article provides a comprehensive survey of
instance retrieval over the last decade. Two broad categories, SIFT-based and
CNN-based methods, are presented. For the former, according to the codebook
size, we organize the literature into using large/medium-sized/small codebooks.
For the latter, we discuss three lines of methods, i.e., using pre-trained or
fine-tuned CNN models, and hybrid methods. The first two perform a single-pass
of an image to the network, while the last category employs a patch-based
feature extraction scheme. This survey presents milestones in modern instance
retrieval, reviews a broad selection of previous works in different categories,
and provides insights on the connection between SIFT and CNN-based methods.
After analyzing and comparing retrieval performance of different categories on
several datasets, we discuss promising directions towards generic and
specialized instance retrieval.
Towards seamless multi-view scene analysis from satellite to street-level
Authors: Sébastien Lefèvre, Devis Tuia, Jan Dirk Wegner, Timothée Produit, Ahmed Samy Nassar
Date: 2017-05-23 00:00:00
Tasks: Change Detection, Earth Observation, object-detection, Object Detection, Scene Understanding

In this paper, we discuss and review how combined multi-view imagery from
satellite to street-level can benefit scene analysis. Numerous works exist that
merge information from remote sensing and images acquired from the ground for
tasks like land cover mapping, object detection, or scene understanding. What
makes the combination of overhead and street-level images challenging, is the
strongly varying viewpoint, different scale, illumination, sensor modality and
time of acquisition. Direct (dense) matching of images on a per-pixel basis is
thus often impossible, and one has to resort to alternative strategies that
will be discussed in this paper. We review recent works that attempt to combine
images taken from the ground and overhead views for purposes like scene
registration, reconstruction, or classification. Three methods that represent
the wide range of potential methods and applications (change detection, image
orientation, and tree cataloging) are described in detail. We show that
cross-fertilization between remote sensing, computer vision and machine
learning is very valuable to make the best of geographic data available from
Earth Observation sensors and ground imagery. Despite its challenges, we
believe that integrating these complementary data sources will lead to major
breakthroughs in Big GeoData.
TwiInsight: Discovering Topics and Sentiments from Social Media Datasets
Authors: Zhengkui Wang, Guangdong Bai, Soumyadeb Chowdhury, Quanqing Xu, Zhi Lin Seow
Date: 2017-05-23 00:00:00
Tasks: Sentiment Analysis

Social media platforms contain a great wealth of information which provides
opportunities for us to explore hidden patterns or unknown correlations, and
understand people's satisfaction with what they are discussing. As one
showcase, in this paper, we present a system, TwiInsight which explores the
insight of Twitter data. Different from other Twitter analysis systems,
TwiInsight automatically extracts the popular topics under different categories
(e.g., healthcare, food, technology, sports and transport) discussed in Twitter
via topic modeling and also identifies the correlated topics across different
categories. Additionally, it also discovers the people's opinions on the tweets
and topics via the sentiment analysis. The system also employs an intuitive and
informative visualization to show the uncovered insight. Furthermore, we also
develop and compare six most popular algorithms - three for sentiment analysis
and three for topic modeling.
Effective injury forecasting in soccer with GPS training data and machine learning
Authors: Alessio Rossi, Luca Pappalardo, Paolo Cintia, Marcello Iaia, Javier Fernandez, Daniel Medina
Date: 2017-05-23 00:00:00
Tasks: BIG-bench Machine Learning

Injuries have a great impact on professional soccer, due to their large
influence on team performance and the considerable costs of rehabilitation for
players. Existing studies in the literature provide just a preliminary
understanding of which factors mostly affect injury risk, while an evaluation
of the potential of statistical models in forecasting injuries is still
missing. In this paper, we propose a multi-dimensional approach to injury
forecasting in professional soccer that is based on GPS measurements and
machine learning. By using GPS tracking technology, we collect data describing
the training workload of players in a professional soccer club during a season.
We then construct an injury forecaster and show that it is both accurate and
interpretable by providing a set of case studies of interest to soccer
practitioners. Our approach opens a novel perspective on injury prevention,
providing a set of simple and practical rules for evaluating and interpreting
the complex relations between injury risk and training performance in
professional soccer.
Multiple Images Recovery Using a Single Affine Transformation
Authors: Bo Jiang, Chris Ding, Bin Luo
Date: 2017-05-23 00:00:00

In many real-world applications, image data often come with noises,
corruptions or large errors. One approach to deal with noise image data is to
use data recovery techniques which aim to recover the true uncorrupted signals
from the observed noise images. In this paper, we first introduce a novel
corruption recovery transformation (CRT) model which aims to recover multiple
(or a collection of) corrupted images using a single affine transformation.
Then, we show that the introduced CRT can be efficiently constructed through
learning from training data. Once CRT is learned, we can recover the true
signals from the new incoming/test corrupted images explicitly. As an
application, we apply our CRT to image recognition task. Experimental results
on six image datasets demonstrate that the proposed CRT model is effective in
recovering noise image data and thus leads to better recognition results.
Contextualizing Citations for Scientific Summarization using Word Embeddings and Domain Knowledge
Authors: Arman Cohan, Nazli Goharian
Date: 2017-05-23 00:00:00
Tasks: Articles, Word Embeddings

Citation texts are sometimes not very informative or in some cases inaccurate
by themselves; they need the appropriate context from the referenced paper to
reflect its exact contributions. To address this problem, we propose an
unsupervised model that uses distributed representation of words as well as
domain knowledge to extract the appropriate context from the reference paper.
Evaluation results show the effectiveness of our model by significantly
outperforming the state-of-the-art. We furthermore demonstrate how an effective
contextualization method results in improving citation-based summarization of
the scientific articles.
Ambiguity set and learning via Bregman and Wasserstein
Authors: Xin Guo, Johnny Hong, Nan Yang
Date: 2017-05-23 00:00:00
Tasks: BIG-bench Machine Learning

Construction of ambiguity set in robust optimization relies on the choice of
divergences between probability distributions. In distribution learning,
choosing appropriate probability distributions based on observed data is
critical for approximating the true distribution. To improve the performance of
machine learning models, there has recently been interest in designing
objective functions based on Lp-Wasserstein distance rather than the classical
Kullback-Leibler (KL) divergence. In this paper, we derive concentration and
asymptotic results using Bregman divergence. We propose a novel asymmetric
statistical divergence called Wasserstein-Bregman divergence as a
generalization of L2-Wasserstein distance. We discuss how these results can be
applied to the construction of ambiguity set in robust optimization.
Non-Convex Weighted Lp Minimization based Group Sparse Representation Framework for Image Denoising
Authors: Qiong Wang, Xinggan Zhang, Yu Wu, Lan Tang, Zhiyuan Zha
Date: 2017-04-05 00:00:00
Tasks: Denoising, Image Denoising

Nonlocal image representation or group sparsity has attracted considerable
interest in various low-level vision tasks and has led to several
state-of-the-art image denoising techniques, such as BM3D, LSSC. In the past,
convex optimization with sparsity-promoting convex regularization was usually
regarded as a standard scheme for estimating sparse signals in noise. However,
using convex regularization can not still obtain the correct sparsity solution
under some practical problems including image inverse problems. In this paper
we propose a non-convex weighted $\ell_p$ minimization based group sparse
representation (GSR) framework for image denoising. To make the proposed scheme
tractable and robust, the generalized soft-thresholding (GST) algorithm is
adopted to solve the non-convex $\ell_p$ minimization problem. In addition, to
improve the accuracy of the nonlocal similar patches selection, an adaptive
patch search (APS) scheme is proposed. Experimental results have demonstrated
that the proposed approach not only outperforms many state-of-the-art denoising
methods such as BM3D and WNNM, but also results in a competitive speed.
Supervised Deep Sparse Coding Networks
Authors: Xiaoxia Sun, Nasser M. Nasrabadi, Trac. D. Tran
Date: 2017-01-29 00:00:00
Tasks: Clustering, General Classification

In this paper, we describe the deep sparse coding network (SCN), a novel deep
network that encodes intermediate representations with nonnegative sparse
coding. The SCN is built upon a number of cascading bottleneck modules, where
each module consists of two sparse coding layers with relatively wide and slim
dictionaries that are specialized to produce high dimensional discriminative
features and low dimensional representations for clustering, respectively.
During training, both the dictionaries and regularization parameters are
optimized with an end-to-end supervised learning algorithm based on multilevel
optimization. Effectiveness of an SCN with seven bottleneck modules is verified
on several popular benchmark datasets. Remarkably, with few parameters to
learn, our SCN achieves 5.81% and 19.93% classification error rate on CIFAR-10
and CIFAR-100, respectively.