pwc-archive/papers-with-abstracts huggingface dataset:
MSR-net:Low-light Image Enhancement Using Deep Convolutional Network
Authors: Liang Shen, Zihan Yue, Fan Feng, Quan Chen, Shihao Liu, Jie Ma
Date: 2017-11-07 00:00:00
Tasks: Image Enhancement, Low-Light Image Enhancement

Images captured in low-light conditions usually suffer from very low
contrast, which increases the difficulty of subsequent computer vision tasks in
a great extent. In this paper, a low-light image enhancement model based on
convolutional neural network and Retinex theory is proposed. Firstly, we show
that multi-scale Retinex is equivalent to a feedforward convolutional neural
network with different Gaussian convolution kernels. Motivated by this fact, we
consider a Convolutional Neural Network(MSR-net) that directly learns an
end-to-end mapping between dark and bright images. Different fundamentally from
existing approaches, low-light image enhancement in this paper is regarded as a
machine learning problem. In this model, most of the parameters are optimized
by back-propagation, while the parameters of traditional models depend on the
artificial setting. Experiments on a number of challenging images reveal the
advantages of our method in comparison with other state-of-the-art methods from
the qualitative and quantitative perspective.
A Survey on Hardware Implementations of Visual Object Trackers
Authors: Al-Hussein A. El-Shafie, S. E. D. Habib
Date: 2017-11-07 00:00:00
Tasks: Object, object-detection, Object Detection, Object Tracking, Survey, Visual Object Tracking

Visual object tracking is an active topic in the computer vision domain with
applications extending over numerous fields. The main sub-tasks required to
build an object tracker (e.g. object detection, feature extraction and object
tracking) are computation-intensive. In addition, real-time operation of the
tracker is indispensable for almost all of its applications. Therefore,
complete hardware or hardware/software co-design approaches are pursued for
better tracker implementations. This paper presents a literature survey of the
hardware implementations of object trackers over the last two decades. Although
several tracking surveys exist in literature, a survey addressing the hardware
implementations of the different trackers is missing. We believe this survey
would fill the gap and complete the picture with the existing surveys of how to
design an efficient tracker and point out the future directions researchers can
follow in this field. We highlight the lack of hardware implementations for
state-of-the-art tracking algorithms as well as for enhanced classical
algorithms. We also stress the need for measuring the tracking performance of
the hardware-based trackers. Additionally, enough details of the hardware-based
trackers need to be provided to allow reasonable comparison between the
different implementations.
A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning
Authors: Marc Lanctot, Vinicius Zambaldi, Audrunas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Perolat, David Silver, Thore Graepel
Date: 2017-11-02 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

To achieve general intelligence, agents must learn how to interact with
others in a shared environment: this is the challenge of multiagent
reinforcement learning (MARL). The simplest form is independent reinforcement
learning (InRL), where each agent treats its experience as part of its
(non-stationary) environment. In this paper, we first observe that policies
learned using InRL can overfit to the other agents' policies during training,
failing to sufficiently generalize during execution. We introduce a new metric,
joint-policy correlation, to quantify this effect. We describe an algorithm for
general MARL, based on approximate best responses to mixtures of policies
generated using deep reinforcement learning, and empirical game-theoretic
analysis to compute meta-strategies for policy selection. The algorithm
generalizes previous ones such as InRL, iterated best response, double oracle,
and fictitious play. Then, we present a scalable implementation which reduces
the memory requirement using decoupled meta-solvers. Finally, we demonstrate
the generality of the resulting policies in two partially observable settings:
gridworld coordination games and poker.
Gaussian Lower Bound for the Information Bottleneck Limit
Authors: Amichai Painsky, Naftali Tishby
Date: 2017-11-07 00:00:00

The Information Bottleneck (IB) is a conceptual method for extracting the
most compact, yet informative, representation of a set of variables, with
respect to the target. It generalizes the notion of minimal sufficient
statistics from classical parametric statistics to a broader
information-theoretic sense. The IB curve defines the optimal trade-off between
representation complexity and its predictive power. Specifically, it is
achieved by minimizing the level of mutual information (MI) between the
representation and the original variables, subject to a minimal level of MI
between the representation and the target. This problem is shown to be in
general NP hard. One important exception is the multivariate Gaussian case, for
which the Gaussian IB (GIB) is known to obtain an analytical closed form
solution, similar to Canonical Correlation Analysis (CCA). In this work we
introduce a Gaussian lower bound to the IB curve; we find an embedding of the
data which maximizes its "Gaussian part", on which we apply the GIB. This
embedding provides an efficient (and practical) representation of any arbitrary
data-set (in the IB sense), which in addition holds the favorable properties of
a Gaussian distribution. Importantly, we show that the optimal Gaussian
embedding is bounded from above by non-linear CCA. This allows a fundamental
limit for our ability to Gaussianize arbitrary data-sets and solve complex
problems by linear methods.
ZipNet-GAN: Inferring Fine-grained Mobile Traffic Patterns via a Generative Adversarial Neural Network
Authors: Chaoyun Zhang, Xi Ouyang, Paul Patras
Date: 2017-11-07 00:00:00
Tasks: Super-Resolution

Large-scale mobile traffic analytics is becoming essential to digital
infrastructure provisioning, public transportation, events planning, and other
domains. Monitoring city-wide mobile traffic is however a complex and costly
process that relies on dedicated probes. Some of these probes have limited
precision or coverage, others gather tens of gigabytes of logs daily, which
independently offer limited insights. Extracting fine-grained patterns involves
expensive spatial aggregation of measurements, storage, and post-processing. In
this paper, we propose a mobile traffic super-resolution technique that
overcomes these problems by inferring narrowly localised traffic consumption
from coarse measurements. We draw inspiration from image processing and design
a deep-learning architecture tailored to mobile networking, which combines
Zipper Network (ZipNet) and Generative Adversarial neural Network (GAN) models.
This enables to uniquely capture spatio-temporal relations between traffic
volume snapshots routinely monitored over broad coverage areas
(`low-resolution') and the corresponding consumption at 0.05 km $^2$ level
(`high-resolution') usually obtained after intensive computation. Experiments
we conduct with a real-world data set demonstrate that the proposed
ZipNet(-GAN) infers traffic consumption with remarkable accuracy and up to
100$\times$ higher granularity as compared to standard probing, while
outperforming existing data interpolation techniques. To our knowledge, this is
the first time super-resolution concepts are applied to large-scale mobile
traffic analysis and our solution is the first to infer fine-grained urban
traffic patterns from coarse aggregates.
Stabilizing Training of Generative Adversarial Networks through Regularization
Authors: Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, Thomas Hofmann
Date: 2017-05-25 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Image Generation

Deep generative models based on Generative Adversarial Networks (GANs) have
demonstrated impressive sample quality but in order to work they require a
careful choice of architecture, parameter initialization, and selection of
hyper-parameters. This fragility is in part due to a dimensional mismatch or
non-overlapping support between the model distribution and the data
distribution, causing their density ratio and the associated f-divergence to be
undefined. We overcome this fundamental limitation and propose a new
regularization approach with low computational cost that yields a stable GAN
training procedure. We demonstrate the effectiveness of this regularizer across
several architectures trained on common benchmark image generation tasks. Our
regularization turns GAN models into reliable building blocks for deep
learning.
Unconstrained Scene Text and Video Text Recognition for Arabic Script
Authors: Mohit Jain, Minesh Mathew, C. V. Jawahar
Date: 2017-11-07 00:00:00
Tasks: Scene Text Recognition

Building robust recognizers for Arabic has always been challenging. We
demonstrate the effectiveness of an end-to-end trainable CNN-RNN hybrid
architecture in recognizing Arabic text in videos and natural scenes. We
outperform previous state-of-the-art on two publicly available video text
datasets - ALIF and ACTIV. For the scene text recognition task, we introduce a
new Arabic scene text dataset and establish baseline results. For scripts like
Arabic, a major challenge in developing robust recognizers is the lack of large
quantity of annotated data. We overcome this by synthesising millions of Arabic
text images from a large vocabulary of Arabic words and phrases. Our
implementation is built on top of the model introduced here [37] which is
proven quite effective for English scene text recognition. The model follows a
segmentation-free, sequence to sequence transcription approach. The network
transcribes a sequence of convolutional features from the input image to a
sequence of target labels. This does away with the need for segmenting input
image into constituent characters/glyphs, which is often difficult for Arabic
script. Further, the ability of RNNs to model contextual dependencies yields
superior recognition results.
Beetle Antennae Search without Parameter Tuning (BAS-WPT) for Multi-objective Optimization
Authors: Xiangyuan Jiang, Shuai Li
Date: 2017-11-07 00:00:00

Beetle antennae search (BAS) is an efficient meta-heuristic algorithm
inspired by foraging behaviors of beetles. This algorithm includes several
parameters for tuning and the existing results are limited to solve single
objective optimization. This work pushes forward the research on BAS by
providing one variant that releases the tuning parameters and is able to handle
multi-objective optimization. This new approach applies normalization to
simplify the original algorithm and uses a penalty function to exploit
infeasible solutions with low constraint violation to solve the constraint
optimization problem. Extensive experimental studies are carried out and the
results reveal efficacy of the proposed approach to constraint handling.
A Tutorial on Canonical Correlation Methods
Authors: Viivi Uurtio, João M. Monteiro, Jaz Kandola, John Shawe-Taylor, Delmiro Fernandez-Reyes, Juho Rousu
Date: 2017-11-07 00:00:00

Canonical correlation analysis is a family of multivariate statistical
methods for the analysis of paired sets of variables. Since its proposition,
canonical correlation analysis has for instance been extended to extract
relations between two sets of variables when the sample size is insufficient in
relation to the data dimensionality, when the relations have been considered to
be non-linear, and when the dimensionality is too large for human
interpretation. This tutorial explains the theory of canonical correlation
analysis including its regularised, kernel, and sparse variants. Additionally,
the deep and Bayesian CCA extensions are briefly reviewed. Together with the
numerical examples, this overview provides a coherent compendium on the
applicability of the variants of canonical correlation analysis. By bringing
together techniques for solving the optimisation problems, evaluating the
statistical significance and generalisability of the canonical correlation
model, and interpreting the relations, we hope that this article can serve as a
hands-on tool for applying canonical correlation methods in data analysis.
Efficient Use of Limited-Memory Accelerators for Linear Learning on Heterogeneous Systems
Authors: Celestine Dünner, Thomas Parnell, Martin Jaggi
Date: 2017-08-17 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: BIG-bench Machine Learning, GPU

We propose a generic algorithmic building block to accelerate training of
machine learning models on heterogeneous compute systems. Our scheme allows to
efficiently employ compute accelerators such as GPUs and FPGAs for the training
of large-scale machine learning models, when the training data exceeds their
memory capacity. Also, it provides adaptivity to any system's memory hierarchy
in terms of size and processing speed. Our technique is built upon novel
theoretical insights regarding primal-dual coordinate methods, and uses duality
gap information to dynamically decide which part of the data should be made
available for fast processing. To illustrate the power of our approach we
demonstrate its performance for training of generalized linear models on a
large-scale dataset exceeding the memory size of a modern GPU, showing an
order-of-magnitude speedup over existing approaches.
First Results from Using Game Refinement Measure and Learning Coefficient in Scrabble
Authors: Kananat Suwanviwatana, Hiroyuki Iida
Date: 2017-11-07 00:00:00

This paper explores the entertainment experience and learning experience in
Scrabble. It proposes a new measure from the educational point of view, which
we call learning coefficient, based on the balance between the learner's skill
and the challenge in Scrabble. Scrabble variants, generated using different
size of board and dictionary, are analyzed with two measures of game refinement
and learning coefficient. The results show that 13x13 Scrabble yields the best
entertainment experience and 15x15 (standard) Scrabble with 4% of original
dictionary size yields the most effective environment for language learners.
Moreover, 15x15 Scrabble with 10% of original dictionary size has a good
balance between entertainment and learning experience.
An EEG-based Image Annotation System
Authors: Viral Parekh, Ramanathan Subramanian, Dipanjan Roy, C. V. Jawahar
Date: 2017-11-07 00:00:00
Tasks: EEG, Electroencephalogram (EEG), ERP

The success of deep learning in computer vision has greatly increased the
need for annotated image datasets. We propose an EEG
(Electroencephalogram)-based image annotation system. While humans can
recognize objects in 20-200 milliseconds, the need to manually label images
results in a low annotation throughput. Our system employs brain signals
captured via a consumer EEG device to achieve an annotation rate of up to 10
images per second. We exploit the P300 event-related potential (ERP) signature
to identify target images during a rapid serial visual presentation (RSVP)
task. We further perform unsupervised outlier removal to achieve an F1-score of
0.88 on the test set. The proposed system does not depend on category-specific
EEG signatures enabling the annotation of any new image category without any
model pre-training.
Distributed Bayesian Piecewise Sparse Linear Models
Authors: Masato Asahara, Ryohei Fujimaki
Date: 2017-11-07 00:00:00
Tasks: CPU, Model Selection

The importance of interpretability of machine learning models has been
increasing due to emerging enterprise predictive analytics, threat of data
privacy, accountability of artificial intelligence in society, and so on.
Piecewise linear models have been actively studied to achieve both accuracy and
interpretability. They often produce competitive accuracy against
state-of-the-art non-linear methods. In addition, their representations (i.e.,
rule-based segmentation plus sparse linear formula) are often preferred by
domain experts. A disadvantage of such models, however, is high computational
cost for simultaneous determinations of the number of "pieces" and cardinality
of each linear predictor, which has restricted their applicability to
middle-scale data sets. This paper proposes a distributed factorized asymptotic
Bayesian (FAB) inference of learning piece-wise sparse linear models on
distributed memory architectures. The distributed FAB inference solves the
simultaneous model selection issue without communicating $O(N)$ data where N is
the number of training samples and achieves linear scale-out against the number
of CPU cores. Experimental results demonstrate that the distributed FAB
inference achieves high prediction accuracy and performance scalability with
both synthetic and benchmark data.
FADO: A Deterministic Detection/Learning Algorithm
Authors: Kristiaan Pelckmans
Date: 2017-11-07 00:00:00
Tasks: Fault Detection, Learning Theory

This paper proposes and studies a detection technique for adversarial
scenarios (dubbed deterministic detection). This technique provides an
alternative detection methodology in case the usual stochastic methods are not
applicable: this can be because the studied phenomenon does not follow a
stochastic sampling scheme, samples are high-dimensional and subsequent
multiple-testing corrections render results overly conservative, sample sizes
are too low for asymptotic results (as e.g. the central limit theorem) to kick
in, or one cannot allow for the small probability of failure inherent to
stochastic approaches. This paper instead designs a method based on insights
from machine learning and online learning theory: this detection algorithm -
named Online FAult Detection (FADO) - comes with theoretical guarantees of its
detection capabilities. A version of the margin is found to regulate the
detection performance of FADO. A precise expression is derived for bounding the
performance, and experimental results are presented assessing the influence of
involved quantities. A case study of scene detection is used to illustrate the
approach. The technology is closely related to the linear perceptron rule,
inherits its computational attractiveness and flexibility towards various
extensions.
Performance Comparison of Intrusion Detection Systems and Application of Machine Learning to Snort System
Authors: Syed Ali Raza Shah, Biju Issac
Date: 2017-10-13 00:00:00
Tasks: BIG-bench Machine Learning, Intrusion Detection

This study investigates the performance of two open source intrusion
detection systems (IDSs) namely Snort and Suricata for accurately detecting the
malicious traffic on computer networks. Snort and Suricata were installed on
two different but identical computers and the performance was evaluated at 10
Gbps network speed. It was noted that Suricata could process a higher speed of
network traffic than Snort with lower packet drop rate but it consumed higher
computational resources. Snort had higher detection accuracy and was thus
selected for further experiments. It was observed that the Snort triggered a
high rate of false positive alarms. To solve this problem a Snort adaptive
plug-in was developed. To select the best performing algorithm for Snort
adaptive plug-in, an empirical study was carried out with different learning
algorithms and Support Vector Machine (SVM) was selected. A hybrid version of
SVM and Fuzzy logic produced a better detection accuracy. But the best result
was achieved using an optimised SVM with firefly algorithm with FPR (false
positive rate) as 8.6% and FNR (false negative rate) as 2.2%, which is a good
result. The novelty of this work is the performance comparison of two IDSs at
10 Gbps and the application of hybrid and optimised machine learning algorithms
to Snort.
Interpreting Convolutional Neural Networks Through Compression
Authors: Reza Abbasi-Asl, Bin Yu
Date: 2017-11-07 00:00:00
Tasks: Object Recognition

Convolutional neural networks (CNNs) achieve state-of-the-art performance in
a wide variety of tasks in computer vision. However, interpreting CNNs still
remains a challenge. This is mainly due to the large number of parameters in
these networks. Here, we investigate the role of compression and particularly
pruning filters in the interpretation of CNNs. We exploit our recently-proposed
greedy structural compression scheme that prunes filters in a trained CNN. In
our compression, the filter importance index is defined as the classification
accuracy reduction (CAR) of the network after pruning that filter. The filters
are then iteratively pruned based on the CAR index. We demonstrate the
interpretability of CAR-compressed CNNs by showing that our algorithm prunes
filters with visually redundant pattern selectivity. Specifically, we show the
importance of shape-selective filters for object recognition, as opposed to
color-selective filters. Out of top 20 CAR-pruned filters in AlexNet, 17 of
them in the first layer and 14 of them in the second layer are color-selective
filters. Finally, we introduce a variant of our CAR importance index that
quantifies the importance of each image class to each CNN filter. We show that
the most and the least important class labels present a meaningful
interpretation of each filter that is consistent with the visualized pattern
selectivity of that filter.
Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks
Authors: Nan Rosemary Ke, Anirudh Goyal, Olexa Bilaniuk, Jonathan Binas, Laurent Charlin, Chris Pal, Yoshua Bengio
Date: 2017-11-07 00:00:00
Proceeding: ICLR 2018 1

A major drawback of backpropagation through time (BPTT) is the difficulty of
learning long-term dependencies, coming from having to propagate credit
information backwards through every single step of the forward computation.
This makes BPTT both computationally impractical and biologically implausible.
For this reason, full backpropagation through time is rarely used on long
sequences, and truncated backpropagation through time is used as a heuristic.
However, this usually leads to biased estimates of the gradient in which longer
term dependencies are ignored. Addressing this issue, we propose an alternative
algorithm, Sparse Attentive Backtracking, which might also be related to
principles used by brains to learn long-term dependencies. Sparse Attentive
Backtracking learns an attention mechanism over the hidden states of the past
and selectively backpropagates through paths with high attention weights. This
allows the model to learn long term dependencies while only backtracking for a
small number of time steps, not just from the recent past but also from
attended relevant past states.