pwc-archive/papers-with-abstracts huggingface dataset:
Auto-Context R-CNN
Authors: Bo Li, Tianfu Wu, Lun Zhang, Rufeng Chu
Date: 2018-07-08 00:00:00
Tasks: Object, object-detection, Object Detection

Region-based convolutional neural networks
(R-CNN)~\cite{fast_rcnn,faster_rcnn,mask_rcnn} have largely dominated object
detection. Operators defined on RoIs (Region of Interests) play an important
role in R-CNNs such as RoIPooling~\cite{fast_rcnn} and
RoIAlign~\cite{mask_rcnn}. They all only utilize information inside RoIs for
RoI prediction, even with their recent deformable
extensions~\cite{deformable_cnn}. Although surrounding context is well-known
for its importance in object detection, it has yet been integrated in R-CNNs in
a flexible and effective way. Inspired by the auto-context
work~\cite{auto_context} and the multi-class object layout
work~\cite{nms_context}, this paper presents a generic context-mining RoI
operator (i.e., \textit{RoICtxMining}) seamlessly integrated in R-CNNs, and the
resulting object detection system is termed \textbf{Auto-Context R-CNN} which
is trained end-to-end. The proposed RoICtxMining operator is a simple yet
effective two-layer extension of the RoIPooling or RoIAlign operator. Centered
at an object-RoI, it creates a $3\times 3$ layout to mine contextual
information adaptively in the $8$ surrounding context regions on-the-fly.
Within each of the $8$ context regions, a context-RoI is mined in term of
discriminative power and its RoIPooling / RoIAlign features are concatenated
with the object-RoI for final prediction. \textit{The proposed Auto-Context
R-CNN is robust to occlusion and small objects, and shows promising
vulnerability for adversarial attacks without being adversarially-trained.} In
experiments, it is evaluated using RoIPooling as the backbone and shows
competitive results on Pascal VOC, Microsoft COCO, and KITTI datasets
(including $6.9\%$ mAP improvements over the R-FCN~\cite{rfcn} method on COCO
\textit{test-dev} dataset and the first place on both KITTI pedestrian and
cyclist detection as of this submission).
Hierarchical stochastic graphlet embedding for graph-based pattern recognition
Authors: Anjan Dutta, Pau Riba, Josep Lladós, Alicia Fornés
Date: 2018-07-08 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Graph Embedding

Despite being very successful within the pattern recognition and machine learning community, graph-based methods are often unusable because of the lack of mathematical operations defined in graph domain. Graph embedding, which maps graphs to a vectorial space, has been proposed as a way to tackle these difficulties enabling the use of standard machine learning techniques. However, it is well known that graph embedding functions usually suffer from the loss of structural information. In this paper, we consider the hierarchical structure of a graph as a way to mitigate this loss of information. The hierarchical structure is constructed by topologically clustering the graph nodes, and considering each cluster as a node in the upper hierarchical level. Once this hierarchical structure is constructed, we consider several configurations to define the mapping into a vector space given a classical graph embedding, in particular, we propose to make use of the Stochastic Graphlet Embedding (SGE). Broadly speaking, SGE produces a distribution of uniformly sampled low to high order graphlets as a way to embed graphs into the vector space. In what follows, the coarse-to-fine structure of a graph hierarchy and the statistics fetched by the SGE complements each other and includes important structural information with varied contexts. Altogether, these two techniques substantially cope with the usual information loss involved in graph embedding techniques, obtaining a more robust graph representation. This fact has been corroborated through a detailed experimental evaluation on various benchmark graph datasets, where we outperform the state-of-the-art methods.
Improving Deep Learning through Automatic Programming
Authors: The-Hien Dang-Ha
Date: 2018-07-08 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, image-classification, Image Classification, Semantic Segmentation, speech-recognition, Speech Recognition

Deep learning and deep architectures are emerging as the best machine
learning methods so far in many practical applications such as reducing the
dimensionality of data, image classification, speech recognition or object
segmentation. In fact, many leading technology companies such as Google,
Microsoft or IBM are researching and using deep architectures in their systems
to replace other traditional models. Therefore, improving the performance of
these models could make a strong impact in the area of machine learning.
However, deep learning is a very fast-growing research domain with many core
methodologies and paradigms just discovered over the last few years. This
thesis will first serve as a short summary of deep learning, which tries to
include all of the most important ideas in this research area. Based on this
knowledge, we suggested, and conducted some experiments to investigate the
possibility of improving the deep learning based on automatic programming
(ADATE). Although our experiments did produce good results, there are still
many more possibilities that we could not try due to limited time as well as
some limitations of the current ADATE version. I hope that this thesis can
promote future work on this topic, especially when the next version of ADATE
comes out. This thesis also includes a short analysis of the power of ADATE
system, which could be useful for other researchers who want to know what it is
capable of.
A Tutorial on Bayesian Optimization
Authors: Peter I. Frazier
Date: 2018-07-08 00:00:00
Tasks: Bayesian Optimization, Hyperparameter Optimization, regression

Bayesian optimization is an approach to optimizing objective functions that
take a long time (minutes or hours) to evaluate. It is best-suited for
optimization over continuous domains of less than 20 dimensions, and tolerates
stochastic noise in function evaluations. It builds a surrogate for the
objective and quantifies the uncertainty in that surrogate using a Bayesian
machine learning technique, Gaussian process regression, and then uses an
acquisition function defined from this surrogate to decide where to sample. In
this tutorial, we describe how Bayesian optimization works, including Gaussian
process regression and three common acquisition functions: expected
improvement, entropy search, and knowledge gradient. We then discuss more
advanced techniques, including running multiple function evaluations in
parallel, multi-fidelity and multi-information source optimization,
expensive-to-evaluate constraints, random environmental conditions, multi-task
Bayesian optimization, and the inclusion of derivative information. We conclude
with a discussion of Bayesian optimization software and future research
directions in the field. Within our tutorial material we provide a
generalization of expected improvement to noisy evaluations, beyond the
noise-free setting where it is more commonly applied. This generalization is
justified by a formal decision-theoretic argument, standing in contrast to
previous ad hoc modifications.
Deeply Supervised Rotation Equivariant Network for Lesion Segmentation in Dermoscopy Images
Authors: Xiaomeng Li, Lequan Yu, Chi-Wing Fu, Pheng-Ann Heng
Date: 2018-07-08 00:00:00
Tasks: Lesion Segmentation, Segmentation, Skin Lesion Segmentation

Automatic lesion segmentation in dermoscopy images is an essential step for
computer-aided diagnosis of melanoma. The dermoscopy images exhibits rotational
and reflectional symmetry, however, this geometric property has not been
encoded in the state-of-the-art convolutional neural networks based skin lesion
segmentation methods. In this paper, we present a deeply supervised rotation
equivariant network for skin lesion segmentation by extending the recent group
rotation equivariant network~\cite{cohen2016group}. Specifically, we propose
the G-upsampling and G-projection operations to adapt the rotation equivariant
classification network for our skin lesion segmentation problem. To further
increase the performance, we integrate the deep supervision scheme into our
proposed rotation equivariant segmentation architecture. The whole framework is
equivariant to input transformations, including rotation and reflection, which
improves the network efficiency and thus contributes to the segmentation
performance. We extensively evaluate our method on the ISIC 2017 skin lesion
challenge dataset. The experimental results show that our rotation equivariant
networks consistently excel the regular counterparts with the same model
complexity under different experimental settings. Our best model achieves
77.23\%(JA) on the test dataset, outperforming the state-of-the-art challenging
methods and further demonstrating the effectiveness of our proposed deeply
supervised rotation equivariant segmentation network. Our best model also
outperforms the state-of-the-art challenging methods, which further demonstrate
the effectiveness of our proposed deeply supervised rotation equivariant
segmentation network.
Revisiting Distillation and Incremental Classifier Learning
Authors: Khurram Javed, Faisal Shafait
Date: 2018-07-08 00:00:00
Tasks: Incremental Learning, Knowledge Distillation

One of the key differences between the learning mechanism of humans and
Artificial Neural Networks (ANNs) is the ability of humans to learn one task at
a time. ANNs, on the other hand, can only learn multiple tasks simultaneously.
Any attempts at learning new tasks incrementally cause them to completely
forget about previous tasks. This lack of ability to learn incrementally,
called Catastrophic Forgetting, is considered a major hurdle in building a true
AI system. In this paper, our goal is to isolate the truly effective existing
ideas for incremental learning from those that only work under certain
conditions. To this end, we first thoroughly analyze the current state of the
art (iCaRL) method for incremental learning and demonstrate that the good
performance of the system is not because of the reasons presented in the
existing literature. We conclude that the success of iCaRL is primarily due to
knowledge distillation and recognize a key limitation of knowledge
distillation, i.e, it often leads to bias in classifiers. Finally, we propose a
dynamic threshold moving algorithm that is able to successfully remove this
bias. We demonstrate the effectiveness of our algorithm on CIFAR100 and MNIST
datasets showing near-optimal results. Our implementation is available at
https://github.com/Khurramjaved96/incremental-learning.
Spatio-Temporal Instance Learning: Action Tubes from Class Supervision
Authors: Pascal Mettes, Cees G. M. Snoek
Date: 2018-07-08 00:00:00
Tasks: Action Localization, Multiple Instance Learning, Reranking, Spatio-Temporal Action Localization, Temporal Action Localization, Weakly Supervised Action Localization

The goal of this work is spatio-temporal action localization in videos, using
only the supervision from video-level class labels. The state-of-the-art casts
this weakly-supervised action localization regime as a Multiple Instance
Learning problem, where instances are a priori computed spatio-temporal
proposals. Rather than disconnecting the spatio-temporal learning from the
training, we propose Spatio-Temporal Instance Learning, which enables action
localization directly from box proposals in video frames. We outline the
assumptions of our model and propose a max-margin objective and optimization
with latent variables that enable spatio-temporal learning of actions from
video labels. We also provide an efficient linking algorithm and two reranking
strategies to facilitate and further improve the action localization.
Experimental evaluation on four action datasets demonstrate the effectiveness
of our approach for localization from weak supervision. Moreover, we show how
to incorporate other supervision levels and mixtures, as a step towards
determining optimal supervision strategies for action localization.
Distillation Techniques for Pseudo-rehearsal Based Incremental Learning
Authors: Haseeb Shah, Khurram Javed, Faisal Shafait
Date: 2018-07-08 00:00:00
Tasks: Incremental Learning, Knowledge Distillation

The ability to learn from incrementally arriving data is essential for any
life-long learning system. However, standard deep neural networks forget the
knowledge about the old tasks, a phenomenon called catastrophic forgetting,
when trained on incrementally arriving data. We discuss the biases in current
Generative Adversarial Networks (GAN) based approaches that learn the
classifier by knowledge distillation from previously trained classifiers. These
biases cause the trained classifier to perform poorly. We propose an approach
to remove these biases by distilling knowledge from the classifier of AC-GAN.
Experiments on MNIST and CIFAR10 show that this method is comparable to current
state of the art rehearsal based approaches. The code for this paper is
available at https://bit.ly/incremental-learning
BALSON: Bayesian Least Squares Optimization with Nonnegative L1-Norm Constraint
Authors: Jiyang Xie, Zhanyu Ma, Guo-Qiang Zhang, Jing-Hao Xue, Jen-Tzung Chien, Zhiqing Lin, Jun Guo
Date: 2018-07-08 00:00:00

A Bayesian approach termed BAyesian Least Squares Optimization with
Nonnegative L1-norm constraint (BALSON) is proposed. The error distribution of
data fitting is described by Gaussian likelihood. The parameter distribution is
assumed to be a Dirichlet distribution. With the Bayes rule, searching for the
optimal parameters is equivalent to finding the mode of the posterior
distribution. In order to explicitly characterize the nonnegative L1-norm
constraint of the parameters, we further approximate the true posterior
distribution by a Dirichlet distribution. We estimate the statistics of the
approximating Dirichlet posterior distribution by sampling methods. Four
sampling methods have been introduced. With the estimated posterior
distributions, the original parameters can be effectively reconstructed in
polynomial fitting problems, and the BALSON framework is found to perform
better than conventional methods.
The modal age of Statistics
Authors: José E. Chacón
Date: 2018-07-08 00:00:00
Tasks: Clustering, regression, Survey

Recently, a number of statistical problems have found an unexpected solution
by inspecting them through a "modal point of view". These include classical
tasks such as clustering or regression. This has led to a renewed interest in
estimation and inference for the mode. This paper offers an extensive survey of
the traditional approaches to mode estimation and explores the consequences of
applying this modern modal methodology to other, seemingly unrelated, fields.
Financial Trading as a Game: A Deep Reinforcement Learning Approach
Authors: Chien Yi Huang
Date: 2018-07-08 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

An automatic program that generates constant profit from the financial market
is lucrative for every market practitioner. Recent advance in deep
reinforcement learning provides a framework toward end-to-end training of such
trading agent. In this paper, we propose an Markov Decision Process (MDP) model
suitable for the financial trading task and solve it with the state-of-the-art
deep recurrent Q-network (DRQN) algorithm. We propose several modifications to
the existing learning algorithm to make it more suitable under the financial
trading setting, namely 1. We employ a substantially small replay memory (only
a few hundreds in size) compared to ones used in modern deep reinforcement
learning algorithms (often millions in size.) 2. We develop an action
augmentation technique to mitigate the need for random exploration by providing
extra feedback signals for all actions to the agent. This enables us to use
greedy policy over the course of learning and shows strong empirical
performance compared to more commonly used epsilon-greedy exploration. However,
this technique is specific to financial trading under a few market assumptions.
3. We sample a longer sequence for recurrent neural network training. A side
product of this mechanism is that we can now train the agent for every T steps.
This greatly reduces training time since the overall computation is down by a
factor of T. We combine all of the above into a complete online learning
algorithm and validate our approach on the spot foreign exchange market.
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
Authors: Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, Yun Fu
Date: 2018-07-08 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image Super-Resolution, Super-Resolution

Convolutional neural network (CNN) depth is of crucial importance for image
super-resolution (SR). However, we observe that deeper networks for image SR
are more difficult to train. The low-resolution inputs and features contain
abundant low-frequency information, which is treated equally across channels,
hence hindering the representational ability of CNNs. To solve these problems,
we propose the very deep residual channel attention networks (RCAN).
Specifically, we propose a residual in residual (RIR) structure to form very
deep network, which consists of several residual groups with long skip
connections. Each residual group contains some residual blocks with short skip
connections. Meanwhile, RIR allows abundant low-frequency information to be
bypassed through multiple skip connections, making the main network focus on
learning high-frequency information. Furthermore, we propose a channel
attention mechanism to adaptively rescale channel-wise features by considering
interdependencies among channels. Extensive experiments show that our RCAN
achieves better accuracy and visual improvements against state-of-the-art
methods.
Model-Free Optimization Using Eagle Perching Optimizer
Authors: Ameer Tamoor Khan, Shuai Li Senior, Predrag S. Stanimirovic, Yinyan Zhang
Date: 2018-07-08 00:00:00
Tasks: Computational Efficiency, model

The paper proposes a novel nature-inspired technique of optimization. It
mimics the perching nature of eagles and uses mathematical formulations to
introduce a new addition to metaheuristic algorithms. The nature of the
proposed algorithm is based on exploration and exploitation. The proposed
algorithm is developed into two versions with some modifications. In the first
phase, it undergoes a rigorous analysis to find out their performance. In the
second phase it is benchmarked using ten functions of two categories; uni-modal
functions and multi-modal functions. In the third phase, we conducted a
detailed analysis of the algorithm by exploiting its controlling units or
variables. In the fourth and last phase, we consider real world optimization
problems with constraints. Both versions of the algorithm show an appreciable
performance, but analysis puts more weight to the modified version. The
competitive analysis shows that the proposed algorithm outperforms the other
tested metaheuristic algorithms. The proposed method has better robustness and
computational efficiency.
Real-time stereo vision-based lane detection system
Authors: Rui Fan, Naim Dahnoun
Date: 2018-07-08 00:00:00
Tasks: CPU, GPU, Lane Detection

The detection of multiple curved lane markings on a non-flat road surface is
still a challenging task for automotive applications. To make an improvement,
the depth information can be used to greatly enhance the robustness of the lane
detection systems. The proposed system in this paper is developed from our
previous work where the dense vanishing point Vp is estimated globally to
assist the detection of multiple curved lane markings. However, the outliers in
the optimal solution may severely affect the accuracy of the least squares
fitting when estimating Vp. Therefore, in this paper we use Random Sample
Consensus to update the inliers and outliers iteratively until the fraction of
the number of inliers versus the total number exceeds our pre-set threshold.
This significantly helps the system to overcome some suddenly changing
conditions. Furthermore, we propose a novel lane position validation approach
which provides a piecewise weight based on Vp and the gradient to reduce the
gradient magnitude of the non-lane candidates. Then, we compute the energy of
each possible solution and select all satisfying lane positions for
visualisation. The proposed system is implemented on a heterogeneous system
which consists of an Intel Core i7-4720HQ CPU and a NVIDIA GTX 970M GPU. A
processing speed of 143 fps has been achieved, which is over 38 times faster
than our previous work. Also, in order to evaluate the detection precision, we
tested 2495 frames with 5361 lanes from the KITTI database (1637 lanes more
than our previous experiment). It is shown that the overall successful
detection rate is improved from 98.7% to 99.5%.
Latent Semantic Analysis Approach for Document Summarization Based on Word Embeddings
Authors: Kamal Al-Sabahi, Zhang Zuping, Yang Kang
Date: 2018-07-08 00:00:00
Tasks: Document Summarization, Word Embeddings

Since the amount of information on the internet is growing rapidly, it is not
easy for a user to find relevant information for his/her query. To tackle this
issue, much attention has been paid to Automatic Document Summarization. The
key point in any successful document summarizer is a good document
representation. The traditional approaches based on word overlapping mostly
fail to produce that kind of representation. Word embedding, distributed
representation of words, has shown an excellent performance that allows words
to match on semantic level. Naively concatenating word embeddings makes the
common word dominant which in turn diminish the representation quality. In this
paper, we employ word embeddings to improve the weighting schemes for
calculating the input matrix of Latent Semantic Analysis method. Two
embedding-based weighting schemes are proposed and then combined to calculate
the values of this matrix. The new weighting schemes are modified versions of
the augment weight and the entropy frequency. The new schemes combine the
strength of the traditional weighting schemes and word embedding. The proposed
approach is experimentally evaluated on three well-known English datasets, DUC
2002, DUC 2004 and Multilingual 2015 Single-document Summarization for English.
The proposed model performs comprehensively better compared to the
state-of-the-art methods, by at least 1% ROUGE points, leading to a conclusion
that it provides a better document representation and a better document summary
as a result.
On the Complexity and Typology of Inflectional Morphological Systems
Authors: Ryan Cotterell, Christo Kirov, Mans Hulden, Jason Eisner
Date: 2018-07-08 00:00:00
Proceeding: TACL 2019 3

We quantify the linguistic complexity of different languages' morphological
systems. We verify that there is an empirical trade-off between paradigm size
and irregularity: a language's inflectional paradigms may be either large in
size or highly irregular, but never both. Our methodology measures paradigm
irregularity as the entropy of the surface realization of a paradigm -- how
hard it is to jointly predict all the surface forms of a paradigm. We estimate
this by a variational approximation. Our measurements are taken on large
morphological paradigms from 31 typologically diverse languages.