pwc-archive/papers-with-abstracts huggingface dataset:
A wavelet frame coefficient total variational model for image restoration
Authors: Wei Wang, Xiang-Gen Xia, Shengli Zhang, Chuanjiang He
Date: 2017-08-25 00:00:00
Tasks: Image Restoration

In this paper, we propose a vector total variation (VTV) of feature image
model for image restoration. The VTV imposes different smoothing powers on
different features (e.g. edges and cartoons) based on choosing various
regularization parameters. Thus, the model can simultaneously preserve edges
and remove noises. Next, the existence of solution for the model is proved and
the split Bregman algorithm is used to solve the model. At last, we use the
wavelet filter banks to explicitly define the feature operator and present some
experimental results to show its advantage over the related methods in both
quality and efficiency.
Enhanced version of AdaBoostM1 with J48 Tree learning method
Authors: Kyongche Kang, Jack Michalak
Date: 2018-02-10 00:00:00
Tasks: BIG-bench Machine Learning, General Classification

Machine Learning focuses on the construction and study of systems that can
learn from data. This is connected with the classification problem, which
usually is what Machine Learning algorithms are designed to solve. When a
machine learning method is used by people with no special expertise in machine
learning, it is important that the method be robust in classification, in the
sense that reasonable performance is obtained with minimal tuning of the
problem at hand. Algorithms are evaluated based on how robust they can classify
the given data. In this paper, we propose a quantifiable measure of robustness,
and describe a particular learning method that is robust according to this
measure in the context of classification problem. We proposed Adaptive Boosting
(AdaBoostM1) with J48(C4.5 tree) as a base learner with tuning weight threshold
(P) and number of iterations (I) for boosting algorithm. To benchmark the
performance, we used the baseline classifier, AdaBoostM1 with Decision Stump as
base learner without tuning parameters. By tuning parameters and using J48 as
base learner, we are able to reduce the overall average error rate ratio
(errorC/errorNB) from 2.4 to 0.9 for development sets of data and 2.1 to 1.2
for evaluation sets of data.
EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
Authors: Pin-Yu Chen, Yash Sharma, huan zhang, Jin-Feng Yi, Cho-Jui Hsieh
Date: 2017-09-13 00:00:00
Tasks: Adversarial Attack, Adversarial Robustness

Recent studies have highlighted the vulnerability of deep neural networks
(DNNs) to adversarial examples - a visually indistinguishable adversarial image
can easily be crafted to cause a well-trained model to misclassify. Existing
methods for crafting adversarial examples are based on $L_2$ and $L_\infty$
distortion metrics. However, despite the fact that $L_1$ distortion accounts
for the total variation and encourages sparsity in the perturbation, little has
been developed for crafting $L_1$-based adversarial examples. In this paper, we
formulate the process of attacking DNNs via adversarial examples as an
elastic-net regularized optimization problem. Our elastic-net attacks to DNNs
(EAD) feature $L_1$-oriented adversarial examples and include the
state-of-the-art $L_2$ attack as a special case. Experimental results on MNIST,
CIFAR10 and ImageNet show that EAD can yield a distinct set of adversarial
examples with small $L_1$ distortion and attains similar attack performance to
the state-of-the-art methods in different attack scenarios. More importantly,
EAD leads to improved attack transferability and complements adversarial
training for DNNs, suggesting novel insights on leveraging $L_1$ distortion in
adversarial machine learning and security implications of DNNs.
Hydra: an Ensemble of Convolutional Neural Networks for Geospatial Land Classification
Authors: Rodrigo Minetto, Mauricio Pamplona Segundo, Sudeep Sarkar
Date: 2018-02-10 00:00:00
Tasks: General Classification

We describe in this paper Hydra, an ensemble of convolutional neural networks
(CNN) for geospatial land classification. The idea behind Hydra is to create an
initial CNN that is coarsely optimized but provides a good starting pointing
for further optimization, which will serve as the Hydra's body. Then, the
obtained weights are fine-tuned multiple times with different augmentation
techniques, crop styles, and classes weights to form an ensemble of CNNs that
represent the Hydra's heads. By doing so, we prompt convergence to different
endpoints, which is a desirable aspect for ensembles. With this framework, we
were able to reduce the training time while maintaining the classification
performance of the ensemble. We created ensembles for our experiments using two
state-of-the-art CNN architectures, ResNet and DenseNet. We have demonstrated
the application of our Hydra framework in two datasets, FMOW and NWPU-RESISC45,
achieving results comparable to the state-of-the-art for the former and the
best reported performance so far for the latter. Code and CNN models are
available at https://github.com/maups/hydra-fmow
Deep learning in radiology: an overview of the concepts and a survey of the state of the art
Authors: Maciej A. Mazurowski, Mateusz Buda, Ashirbani Saha, Mustafa R. Bashir
Date: 2018-02-10 00:00:00
Tasks: Deep Learning

Deep learning is a branch of artificial intelligence where networks of simple
interconnected units are used to extract patterns from data in order to solve
complex problems. Deep learning algorithms have shown groundbreaking
performance in a variety of sophisticated tasks, especially those related to
images. They have often matched or exceeded human performance. Since the
medical field of radiology mostly relies on extracting useful information from
images, it is a very natural application area for deep learning, and research
in this area has rapidly grown in recent years. In this article, we review the
clinical reality of radiology and discuss the opportunities for application of
deep learning algorithms. We also introduce basic concepts of deep learning
including convolutional neural networks. Then, we present a survey of the
research in deep learning applied to radiology. We organize the studies by the
types of specific tasks that they attempt to solve and review the broad range
of utilized deep learning algorithms. Finally, we briefly discuss opportunities
and challenges for incorporating deep learning in the radiology practice of the
future.
On-device Scalable Image-based Localization via Prioritized Cascade Search and Fast One-Many RANSAC
Authors: Ngoc-Trung Tran, Dang-Khoa Le Tan, Anh-Dzung Doan, Thanh-Toan Do, Tuan-Anh Bui, Mengxuan Tan, Ngai-Man Cheung
Date: 2018-02-10 00:00:00
Tasks: Image-Based Localization, Image Retrieval, Pose Estimation, Retrieval

We present the design of an entire on-device system for large-scale urban
localization using images. The proposed design integrates compact image
retrieval and 2D-3D correspondence search to estimate the location in extensive
city regions. Our design is GPS agnostic and does not require network
connection. In order to overcome the resource constraints of mobile devices, we
propose a system design that leverages the scalability advantage of image
retrieval and accuracy of 3D model-based localization. Furthermore, we propose
a new hashing-based cascade search for fast computation of 2D-3D
correspondences. In addition, we propose a new one-many RANSAC for accurate
pose estimation. The new one-many RANSAC addresses the challenge of repetitive
building structures (e.g. windows, balconies) in urban localization. Extensive
experiments demonstrate that our 2D-3D correspondence search achieves
state-of-the-art localization accuracy on multiple benchmark datasets.
Furthermore, our experiments on a large Google Street View (GSV) image dataset
show the potential of large-scale localization entirely on a typical mobile
device.
Scalable Peaceman-Rachford Splitting Method with Proximal Terms
Authors: Sen Na, Mingyuan Ma, Mladen Kolar
Date: 2017-11-14 00:00:00

Along with developing of Peaceman-Rachford Splittling Method (PRSM), many
batch algorithms based on it have been studied very deeply. But almost no
algorithm focused on the performance of stochastic version of PRSM. In this
paper, we propose a new stochastic algorithm based on PRSM, prove its
convergence rate in ergodic sense, and test its performance on both artificial
and real data. We show that our proposed algorithm, Stochastic Scalable PRSM
(SS-PRSM), enjoys the $O(1/K)$ convergence rate, which is the same as those
newest stochastic algorithms that based on ADMM but faster than general
Stochastic ADMM (which is $O(1/\sqrt{K})$). Our algorithm also owns wide
flexibility, outperforms many state-of-the-art stochastic algorithms coming
from ADMM, and has low memory cost in large-scale splitting optimization
problems.
Path Consistency Learning in Tsallis Entropy Regularized MDPs
Authors: Ofir Nachum, Yin-Lam Chow, Mohammad Ghavamzadeh
Date: 2018-02-10 00:00:00
Proceeding: ICML 2018 7
Tasks: Reinforcement Learning

We study the sparse entropy-regularized reinforcement learning (ERL) problem
in which the entropy term is a special form of the Tsallis entropy. The optimal
policy of this formulation is sparse, i.e.,~at each state, it has non-zero
probability for only a small number of actions. This addresses the main
drawback of the standard Shannon entropy-regularized RL (soft ERL) formulation,
in which the optimal policy is softmax, and thus, may assign a non-negligible
probability mass to non-optimal actions. This problem is aggravated as the
number of actions is increased. In this paper, we follow the work of Nachum et
al. (2017) in the soft ERL setting, and propose a class of novel path
consistency learning (PCL) algorithms, called {\em sparse PCL}, for the sparse
ERL problem that can work with both on-policy and off-policy data. We first
derive a {\em sparse consistency} equation that specifies a relationship
between the optimal value function and policy of the sparse ERL along any
system trajectory. Crucially, a weak form of the converse is also true, and we
quantify the sub-optimality of a policy which satisfies sparse consistency, and
show that as we increase the number of actions, this sub-optimality is better
than that of the soft ERL optimal policy. We then use this result to derive the
sparse PCL algorithms. We empirically compare sparse PCL with its soft
counterpart, and show its advantage, especially in problems with a large number
of actions.
Local Contrast Learning
Authors: Chuanyun Xu, Yang Zhang, Xin Feng, YongXing Ge, Yihao Zhang, Jianwu Long
Date: 2018-02-10 00:00:00
Tasks: General Classification, Small Data Image Classification

Learning a deep model from small data is yet an opening and challenging
problem. We focus on one-shot classification by deep learning approach based on
a small quantity of training samples. We proposed a novel deep learning
approach named Local Contrast Learning (LCL) based on the key insight about a
human cognitive behavior that human recognizes the objects in a specific
context by contrasting the objects in the context or in her/his memory. LCL is
used to train a deep model that can contrast the recognizing sample with a
couple of contrastive samples randomly drawn and shuffled. On one-shot
classification task on Omniglot, the deep model based LCL with 122 layers and
1.94 millions of parameters, which was trained on a tiny dataset with only 60
classes and 20 samples per class, achieved the accuracy 97.99% that outperforms
human and state-of-the-art established by Bayesian Program Learning (BPL)
trained on 964 classes. LCL is a fundamental idea which can be applied to
alleviate parametric model's overfitting resulted by lack of training samples.
Generative Adversarial Networks and Probabilistic Graph Models for Hyperspectral Image Classification
Authors: Zilong Zhong, Jonathan Li
Date: 2018-02-10 00:00:00
Tasks: Classification, General Classification, Hyperspectral Image Classification, image-classification, Image Classification

High spectral dimensionality and the shortage of annotations make
hyperspectral image (HSI) classification a challenging problem. Recent studies
suggest that convolutional neural networks can learn discriminative spatial
features, which play a paramount role in HSI interpretation. However, most of
these methods ignore the distinctive spectral-spatial characteristic of
hyperspectral data. In addition, a large amount of unlabeled data remains an
unexploited gold mine for efficient data use. Therefore, we proposed an
integration of generative adversarial networks (GANs) and probabilistic
graphical models for HSI classification. Specifically, we used a
spectral-spatial generator and a discriminator to identify land cover
categories of hyperspectral cubes. Moreover, to take advantage of a large
amount of unlabeled data, we adopted a conditional random field to refine the
preliminary classification results generated by GANs. Experimental results
obtained using two commonly studied datasets demonstrate that the proposed
framework achieved encouraging classification accuracy using a small number of
data for training.
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
Authors: Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, Song Han
Date: 2018-02-10 00:00:00
Proceeding: ECCV 2018 9
Tasks: AutoML, GPU, Model Compression, Neural Architecture Search, Reinforcement Learning

Model compression is a critical technique to efficiently deploy neural
network models on mobile devices which have limited computation resources and
tight power budgets. Conventional model compression techniques rely on
hand-crafted heuristics and rule-based policies that require domain experts to
explore the large design space trading off among model size, speed, and
accuracy, which is usually sub-optimal and time-consuming. In this paper, we
propose AutoML for Model Compression (AMC) which leverage reinforcement
learning to provide the model compression policy. This learning-based
compression policy outperforms conventional rule-based compression policy by
having higher compression ratio, better preserving the accuracy and freeing
human labor. Under 4x FLOPs reduction, we achieved 2.7% better accuracy than
the handcrafted model compression policy for VGG-16 on ImageNet. We applied
this automated, push-the-button compression pipeline to MobileNet and achieved
1.81x speedup of measured inference latency on an Android phone and 1.43x
speedup on the Titan XP GPU, with only 0.1% loss of ImageNet Top-1 accuracy.
MAGAN: Aligning Biological Manifolds
Authors: Matthew Amodio, Smita Krishnaswamy
Date: 2018-02-10 00:00:00
Proceeding: ICML 2018 7

It is increasingly common in many types of natural and physical systems
(especially biological systems) to have different types of measurements
performed on the same underlying system. In such settings, it is important to
align the manifolds arising from each measurement in order to integrate such
data and gain an improved picture of the system. We tackle this problem using
generative adversarial networks (GANs). Recently, GANs have been utilized to
try to find correspondences between sets of samples. However, these GANs are
not explicitly designed for proper alignment of manifolds. We present a new GAN
called the Manifold-Aligning GAN (MAGAN) that aligns two manifolds such that
related points in each measurement space are aligned together. We demonstrate
applications of MAGAN in single-cell biology in integrating two different
measurement types together. In our demonstrated examples, cells from the same
tissue are measured with both genomic (single-cell RNA-sequencing) and
proteomic (mass cytometry) technologies. We show that the MAGAN successfully
aligns them such that known correlations between measured markers are improved
compared to other recently proposed models.
Invertible Autoencoder for domain adaptation
Authors: Yunfei Teng, Anna Choromanska, Mariusz Bojarski
Date: 2018-02-10 00:00:00
Tasks: Autonomous Driving, Domain Adaptation, Image-to-Image Translation, Translation, Unsupervised Image-To-Image Translation

The unsupervised image-to-image translation aims at finding a mapping between
the source ($A$) and target ($B$) image domains, where in many applications
aligned image pairs are not available at training. This is an ill-posed
learning problem since it requires inferring the joint probability distribution
from marginals. Joint learning of coupled mappings $F_{AB}: A \rightarrow B$
and $F_{BA}: B \rightarrow A$ is commonly used by the state-of-the-art methods,
like CycleGAN [Zhu et al., 2017], to learn this translation by introducing
cycle consistency requirement to the learning problem, i.e. $F_{AB}(F_{BA}(B))
\approx B$ and $F_{BA}(F_{AB}(A)) \approx A$. Cycle consistency enforces the
preservation of the mutual information between input and translated images.
However, it does not explicitly enforce $F_{BA}$ to be an inverse operation to
$F_{AB}$. We propose a new deep architecture that we call invertible
autoencoder (InvAuto) to explicitly enforce this relation. This is done by
forcing an encoder to be an inverted version of the decoder, where
corresponding layers perform opposite mappings and share parameters. The
mappings are constrained to be orthonormal. The resulting architecture leads to
the reduction of the number of trainable parameters (up to $2$ times). We
present image translation results on benchmark data sets and demonstrate
state-of-the art performance of our approach. Finally, we test the proposed
domain adaptation method on the task of road video conversion. We demonstrate
that the videos converted with InvAuto have high quality and show that the
NVIDIA neural-network-based end-to-end learning system for autonomous driving,
known as PilotNet, trained on real road videos performs well when tested on the
converted ones.
Generalization of an Upper Bound on the Number of Nodes Needed to Achieve Linear Separability
Authors: Marjolein Troost, Katja Seeliger, Marcel van Gerven
Date: 2018-02-10 00:00:00

An important issue in neural network research is how to choose the number of
nodes and layers such as to solve a classification problem. We provide new
intuitions based on earlier results by An et al. (2015) by deriving an upper
bound on the number of nodes in networks with two hidden layers such that
linear separability can be achieved. Concretely, we show that if the data can
be described in terms of N finite sets and the used activation function f is
non-constant, increasing and has a left asymptote, we can derive how many nodes
are needed to linearly separate these sets. This will be an upper bound that
depends on the structure of the data. This structure can be analyzed using an
algorithm. For the leaky rectified linear activation function, we prove
separately that under some conditions on the slope, the same number of layers
and nodes as for the aforementioned activation functions is sufficient. We
empirically validate our claims.
Small nonlinearities in activation functions create bad local minima in neural networks
Authors: Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Date: 2018-02-10 00:00:00
Proceeding: ICLR 2019 5

We investigate the loss surface of neural networks. We prove that even for one-hidden-layer networks with "slightest" nonlinearity, the empirical risks have spurious local minima in most cases. Our results thus indicate that in general "no spurious local minima" is a property limited to deep linear networks, and insights obtained from linear networks may not be robust. Specifically, for ReLU(-like) networks we constructively prove that for almost all practical datasets there exist infinitely many local minima. We also present a counterexample for more general activations (sigmoid, tanh, arctan, ReLU, etc.), for which there exists a bad local minimum. Our results make the least restrictive assumptions relative to existing results on spurious local optima in neural networks. We complete our discussion by presenting a comprehensive characterization of global optimality for deep linear networks, which unifies other results on this topic.
An LSTM Recurrent Network for Step Counting
Authors: Ziyi Chen
Date: 2018-02-10 00:00:00

Smartphones with sensors such as accelerometer and gyroscope can be used as
pedometers and navigators. In this paper, we propose to use an LSTM recurrent
network for counting the number of steps taken by both blind and sighted users,
based on an annotated smartphone sensor dataset, WeAllWork. The models were
trained separately for sighted people, blind people with a long cane or a guide
dog for Leave-One-Out training modality. It achieved 5% overcount and
undercount rate.
Conditional Adversarial Domain Adaptation
Authors: Mingsheng Long, Zhangjie Cao, Jian-Min Wang, Michael. I. Jordan
Date: 2017-05-26 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Domain Adaptation, General Classification

Adversarial learning has been embedded into deep networks to learn
disentangled and transferable representations for domain adaptation. Existing
adversarial domain adaptation methods may not effectively align different
domains of multimodal distributions native in classification problems. In this
paper, we present conditional adversarial domain adaptation, a principled
framework that conditions the adversarial adaptation models on discriminative
information conveyed in the classifier predictions. Conditional domain
adversarial networks (CDANs) are designed with two novel conditioning
strategies: multilinear conditioning that captures the cross-covariance between
feature representations and classifier predictions to improve the
discriminability, and entropy conditioning that controls the uncertainty of
classifier predictions to guarantee the transferability. With theoretical
guarantees and a few lines of codes, the approach has exceeded state-of-the-art
results on five datasets.
A Continuation Method for Discrete Optimization and its Application to Nearest Neighbor Classification
Authors: Ali Shameli, Yasin Abbasi-Yadkori
Date: 2018-02-10 00:00:00
Tasks: General Classification

The continuation method is a popular approach in non-convex optimization and
computer vision. The main idea is to start from a simple function that can be
minimized efficiently, and gradually transform it to the more complicated
original objective function. The solution of the simpler problem is used as the
starting point to solve the original problem. We show a continuation method for
discrete optimization problems. Ideally, we would like the evolved function to
be hill-climbing friendly and to have the same global minima as the original
function. We show that the proposed continuation method is the best affine
approximation of a transformation that is guaranteed to transform the function
to a hill-climbing friendly function and to have the same global minima.
  We show the effectiveness of the proposed technique in the problem of nearest
neighbor classification. Although nearest neighbor methods are often
competitive in terms of sample efficiency, the computational complexity in the
test phase has been a major obstacle in their applicability in big data
problems. Using the proposed continuation method, we show an improved
graph-based nearest neighbor algorithm. The method is readily understood and
easy to implement. We show how the computational complexity of the method in
the test phase scales gracefully with the size of the training set, a property
that is particularly important in big data applications.