pwc-archive/papers-with-abstracts huggingface dataset:
Curriculum Learning for Multi-Task Classification of Visual Attributes
Authors: Nikolaos Sarafianos, Theodore Giannakopoulos, Christophoros Nikou, Ioannis A. Kakadiaris
Date: 2017-08-29 00:00:00
Tasks: Attribute, Classification, General Classification, Image Description, Multi-Task Learning

Visual attributes, from simple objects (e.g., backpacks, hats) to
soft-biometrics (e.g., gender, height, clothing) have proven to be a powerful
representational approach for many applications such as image description and
human identification. In this paper, we introduce a novel method to combine the
advantages of both multi-task and curriculum learning in a visual attribute
classification framework. Individual tasks are grouped based on their
correlation so that two groups of strongly and weakly correlated tasks are
formed. The two groups of tasks are learned in a curriculum learning setup by
transferring the acquired knowledge from the strongly to the weakly correlated.
The learning process within each group though, is performed in a multi-task
classification setup. The proposed method learns better and converges faster
than learning all the tasks in a typical multi-task learning paradigm. We
demonstrate the effectiveness of our approach on the publicly available, SoBiR,
VIPeR and PETA datasets and report state-of-the-art results across the board.
Machine Learning Approach for Detection of nonTor Traffic
Authors: Elike Hodo, Xavier Bellekens, Ephraim Iorkyase, Andrew Hamilton, Christos Tachtatzis, Robert Atkinson
Date: 2017-08-29 00:00:00
Tasks: BIG-bench Machine Learning, Intrusion Detection

Intrusion detection has attracted a considerable interest from researchers
and industries. After many years of research the community still faces the
problem of building reliable and efficient intrusion detection systems (IDS)
capable of handling large quantities of data with changing patterns in real
time situations. The Tor network is popular in providing privacy and security
to end user by anonymising the identity of internet users connecting through a
series of tunnels and nodes. This work focuses on the classification of Tor
traffic and nonTor traffic to expose the activities within Tor traffic that
minimizes the protection of users. A study to compare the reliability and
efficiency of Artificial Neural Network and Support vector machine in detecting
nonTor traffic in UNB-CIC Tor Network Traffic dataset is presented in this
paper. The results are analysed based on the overall accuracy, detection rate
and false positive rate of the two algorithms. Experimental results show that
both algorithms could detect nonTor traffic in the dataset. A hybrid Artificial
neural network proved a better classifier than SVM in detecting nonTor traffic
in UNB-CIC Tor Network Traffic dataset.
Molecular De Novo Design through Deep Reinforcement Learning
Authors: Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, Hongming Chen
Date: 2017-04-25 00:00:00
Tasks: Activity Prediction, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This work introduces a method to tune a sequence-based generative model for
molecular de novo design that through augmented episodic likelihood can learn
to generate structures with certain specified desirable properties. We
demonstrate how this model can execute a range of tasks such as generating
analogues to a query structure and generating compounds predicted to be active
against a biological target. As a proof of principle, the model is first
trained to generate molecules that do not contain sulphur. As a second example,
the model is trained to generate analogues to the drug Celecoxib, a technique
that could be used for scaffold hopping or library expansion starting from a
single molecule. Finally, when tuning the model towards generating compounds
predicted to be active against the dopamine receptor type 2, the model
generates structures of which more than 95% are predicted to be active,
including experimentally confirmed actives that have not been included in
either the generative model nor the activity prediction model.
Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks
Authors: Lucas Fidon, Wenqi Li, Luis C. Garcia-Peraza-Herrera, Jinendra Ekanayake, Neil Kitchen, Sebastien Ourselin, Tom Vercauteren
Date: 2017-07-03 00:00:00
Tasks: Segmentation

The Dice score is widely used for binary segmentation due to its robustness
to class imbalance. Soft generalisations of the Dice score allow it to be used
as a loss function for training convolutional neural networks (CNN). Although
CNNs trained using mean-class Dice score achieve state-of-the-art results on
multi-class segmentation, this loss function does neither take advantage of
inter-class relationships nor multi-scale information. We argue that an
improved loss function should balance misclassifications to favour predictions
that are semantically meaningful. This paper investigates these issues in the
context of multi-class brain tumour segmentation. Our contribution is
threefold. 1) We propose a semantically-informed generalisation of the Dice
score for multi-class segmentation based on the Wasserstein distance on the
probabilistic label space. 2) We propose a holistic CNN that embeds spatial
information at multiple scales with deep supervision. 3) We show that the joint
use of holistic CNNs and generalised Wasserstein Dice scores achieves
segmentations that are more semantically meaningful for brain tumour
segmentation.
Solving Optimization Problems by the Public Goods Game
Authors: Marco Alberto Javarone
Date: 2016-04-07 00:00:00
Tasks: Combinatorial Optimization, Traveling Salesman Problem

We introduce a method based on the Public Goods Game for solving optimization
tasks. In particular, we focus on the Traveling Salesman Problem, i.e. a
NP-hard problem whose search space exponentially grows increasing the number of
cities. The proposed method considers a population whose agents are provided
with a random solution to the given problem. In doing so, agents interact by
playing the Public Goods Game using the fitness of their solution as currency
of the game. Notably, agents with better solutions provide higher
contributions, while those with lower ones tend to imitate the solution of
richer agents for increasing their fitness. Numerical simulations show that the
proposed method allows to compute exact solutions, and suboptimal ones, in the
considered search spaces. As result, beyond to propose a new heuristic for
combinatorial optimization problems, our work aims to highlight the
potentiality of evolutionary game theory beyond its current horizons.
Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learning
Authors: Jeremias Sulam, Vardan Papyan, Yaniv Romano, Michael Elad
Date: 2017-08-29 00:00:00
Tasks: Dictionary Learning

The recently proposed Multi-Layer Convolutional Sparse Coding (ML-CSC) model,
consisting of a cascade of convolutional sparse layers, provides a new
interpretation of Convolutional Neural Networks (CNNs). Under this framework,
the computation of the forward pass in a CNN is equivalent to a pursuit
algorithm aiming to estimate the nested sparse representation vectors -- or
feature maps -- from a given input signal. Despite having served as a pivotal
connection between CNNs and sparse modeling, a deeper understanding of the
ML-CSC is still lacking: there are no pursuit algorithms that can serve this
model exactly, nor are there conditions to guarantee a non-empty model. While
one can easily obtain signals that approximately satisfy the ML-CSC
constraints, it remains unclear how to simply sample from the model and, more
importantly, how one can train the convolutional filters from real data.
  In this work, we propose a sound pursuit algorithm for the ML-CSC model by
adopting a projection approach. We provide new and improved bounds on the
stability of the solution of such pursuit and we analyze different practical
alternatives to implement this in practice. We show that the training of the
filters is essential to allow for non-trivial signals in the model, and we
derive an online algorithm to learn the dictionaries from real data,
effectively resulting in cascaded sparse convolutional layers. Last, but not
least, we demonstrate the applicability of the ML-CSC model for several
applications in an unsupervised setting, providing competitive results. Our
work represents a bridge between matrix factorization, sparse dictionary
learning and sparse auto-encoders, and we analyze these connections in detail.
Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization
Authors: Luis Muñoz-González, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C. Lupu, Fabio Roli
Date: 2017-08-29 00:00:00
Tasks: Data Poisoning, Deep Learning, Handwritten Digit Recognition, Malware Detection

A number of online services nowadays rely upon machine learning to extract
valuable information from data collected in the wild. This exposes learning
algorithms to the threat of data poisoning, i.e., a coordinate attack in which
a fraction of the training data is controlled by the attacker and manipulated
to subvert the learning process. To date, these attacks have been devised only
against a limited class of binary learning algorithms, due to the inherent
complexity of the gradient-based procedure used to optimize the poisoning
points (a.k.a. adversarial training examples). In this work, we rst extend the
de nition of poisoning attacks to multiclass problems. We then propose a novel
poisoning algorithm based on the idea of back-gradient optimization, i.e., to
compute the gradient of interest through automatic di erentiation, while also
reversing the learning procedure to drastically reduce the attack complexity.
Compared to current poisoning strategies, our approach is able to target a
wider class of learning algorithms, trained with gradient- based procedures,
including neural networks and deep learning architectures. We empirically
evaluate its e ectiveness on several application examples, including spam
ltering, malware detection, and handwritten digit recognition. We nally show
that, similarly to adversarial test examples, adversarial training examples can
also be transferred across di erent learning algorithms.
Performance Guaranteed Network Acceleration via High-Order Residual Quantization
Authors: Zefan Li, Bingbing Ni, Wenjun Zhang, Xiaokang Yang, Wen Gao
Date: 2017-08-29 00:00:00
Proceeding: ICCV 2017 10
Tasks: Binarization, Quantization, Vocal Bursts Intensity Prediction

Input binarization has shown to be an effective way for network acceleration.
However, previous binarization scheme could be regarded as simple pixel-wise
thresholding operations (i.e., order-one approximation) and suffers a big
accuracy loss. In this paper, we propose a highorder binarization scheme, which
achieves more accurate approximation while still possesses the advantage of
binary operation. In particular, the proposed scheme recursively performs
residual quantization and yields a series of binary input images with
decreasing magnitude scales. Accordingly, we propose high-order binary
filtering and gradient propagation operations for both forward and backward
computations. Theoretical analysis shows approximation error guarantee property
of proposed method. Extensive experimental results demonstrate that the
proposed scheme yields great recognition accuracy while being accelerated.
Performance Analysis of Open Source Machine Learning Frameworks for Various Parameters in Single-Threaded and Multi-Threaded Modes
Authors: Yuriy Kochura, Sergii Stirenko, Oleg Alienin, Michail Novotarskiy, Yuri Gordienko
Date: 2017-08-29 00:00:00
Tasks: BIG-bench Machine Learning, CPU, GPU, Hyperparameter Optimization

The basic features of some of the most versatile and popular open source
frameworks for machine learning (TensorFlow, Deep Learning4j, and H2O) are
considered and compared. Their comparative analysis was performed and
conclusions were made as to the advantages and disadvantages of these
platforms. The performance tests for the de facto standard MNIST data set were
carried out on H2O framework for deep learning algorithms designed for CPU and
GPU platforms for single-threaded and multithreaded modes of operation Also, we
present the results of testing neural networks architectures on H2O platform
for various activation functions, stopping metrics, and other parameters of
machine learning algorithm. It was demonstrated for the use case of MNIST
database of handwritten digits in single-threaded mode that blind selection of
these parameters can hugely increase (by 2-3 orders) the runtime without the
significant increase of precision. This result can have crucial influence for
optimization of available and new machine learning methods, especially for
image recognition problems.
Fixed-point Factorized Networks
Authors: Peisong Wang, Jian Cheng
Date: 2016-11-07 00:00:00
Proceeding: CVPR 2017 7
Tasks: General Classification

In recent years, Deep Neural Networks (DNN) based methods have achieved
remarkable performance in a wide range of tasks and have been among the most
powerful and widely used techniques in computer vision. However, DNN-based
methods are both computational-intensive and resource-consuming, which hinders
the application of these methods on embedded systems like smart phones. To
alleviate this problem, we introduce a novel Fixed-point Factorized Networks
(FFN) for pretrained models to reduce the computational complexity as well as
the storage requirement of networks. The resulting networks have only weights
of -1, 0 and 1, which significantly eliminates the most resource-consuming
multiply-accumulate operations (MACs). Extensive experiments on large-scale
ImageNet classification task show the proposed FFN only requires one-thousandth
of multiply operations with comparable accuracy.
Implications of Decentralized Q-learning Resource Allocation in Wireless Networks
Authors: Francesc Wilhelmi, Boris Bellalta, Cristina Cano, Anders Jonsson
Date: 2017-05-30 00:00:00
Tasks: Q-Learning, Reinforcement Learning

Reinforcement Learning is gaining attention by the wireless networking
community due to its potential to learn good-performing configurations only
from the observed results. In this work we propose a stateless variation of
Q-learning, which we apply to exploit spatial reuse in a wireless network. In
particular, we allow networks to modify both their transmission power and the
channel used solely based on the experienced throughput. We concentrate in a
completely decentralized scenario in which no information about neighbouring
nodes is available to the learners. Our results show that although the
algorithm is able to find the best-performing actions to enhance aggregate
throughput, there is high variability in the throughput experienced by the
individual networks. We identify the cause of this variability as the
adversarial setting of our setup, in which the most played actions provide
intermittent good/poor performance depending on the neighbouring decisions. We
also evaluate the effect of the intrinsic learning parameters of the algorithm
on this variability.
Comparing Human and Machine Errors in Conversational Speech Transcription
Authors: Andreas Stolcke, Jasha Droppo
Date: 2017-08-29 00:00:00

Recent work in automatic recognition of conversational telephone speech (CTS)
has achieved accuracy levels comparable to human transcribers, although there
is some debate how to precisely quantify human performance on this task, using
the NIST 2000 CTS evaluation set. This raises the question what systematic
differences, if any, may be found differentiating human from machine
transcription errors. In this paper we approach this question by comparing the
output of our most accurate CTS recognition system to that of a standard speech
transcription vendor pipeline. We find that the most frequent substitution,
deletion and insertion error types of both outputs show a high degree of
overlap. The only notable exception is that the automatic recognizer tends to
confuse filled pauses ("uh") and backchannel acknowledgments ("uhhuh"). Humans
tend not to make this error, presumably due to the distinctive and opposing
pragmatic functions attached to these words. Furthermore, we quantify the
correlation between human and machine errors at the speaker level, and
investigate the effect of speaker overlap between training and test data.
Finally, we report on an informal "Turing test" asking humans to discriminate
between automatic and human transcription error cases.
EC3: Combining Clustering and Classification for Ensemble Learning
Authors: Tanmoy Chakraborty
Date: 2017-08-29 00:00:00
Tasks: Classification, Clustering, Ensemble Learning, General Classification, Multi-class Classification

Classification and clustering algorithms have been proved to be successful
individually in different contexts. Both of them have their own advantages and
limitations. For instance, although classification algorithms are more powerful
than clustering methods in predicting class labels of objects, they do not
perform well when there is a lack of sufficient manually labeled reliable data.
On the other hand, although clustering algorithms do not produce label
information for objects, they provide supplementary constraints (e.g., if two
objects are clustered together, it is more likely that the same label is
assigned to both of them) that one can leverage for label prediction of a set
of unknown objects. Therefore, systematic utilization of both these types of
algorithms together can lead to better prediction performance. In this paper,
We propose a novel algorithm, called EC3 that merges classification and
clustering together in order to support both binary and multi-class
classification. EC3 is based on a principled combination of multiple
classification and multiple clustering methods using an optimization function.
We theoretically show the convexity and optimality of the problem and solve it
by block coordinate descent method. We additionally propose iEC3, a variant of
EC3 that handles imbalanced training data. We perform an extensive experimental
analysis by comparing EC3 and iEC3 with 14 baseline methods (7 well-known
standalone classifiers, 5 ensemble classifiers, and 2 existing methods that
merge classification and clustering) on 13 standard benchmark datasets. We show
that our methods outperform other baselines for every single dataset, achieving
at most 10% higher AUC. Moreover our methods are faster (1.21 times faster than
the best baseline), more resilient to noise and class imbalance than the best
baseline method.
CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices
Authors: Caiwen Ding, Siyu Liao, Yanzhi Wang, Zhe Li, Ning Liu, Youwei Zhuo, Chao Wang, Xuehai Qian, Yu Bai, Geng Yuan, Xiaolong Ma, Yi-Peng Zhang, Jian Tang, Qinru Qiu, Xue Lin, Bo Yuan
Date: 2017-08-29 00:00:00

Large-scale deep neural networks (DNNs) are both compute and memory
intensive. As the size of DNNs continues to grow, it is critical to improve the
energy efficiency and performance while maintaining accuracy. For DNNs, the
model size is an important factor affecting performance, scalability and energy
efficiency. Weight pruning achieves good compression ratios but suffers from
three drawbacks: 1) the irregular network structure after pruning; 2) the
increased training complexity; and 3) the lack of rigorous guarantee of
compression ratio and inference accuracy. To overcome these limitations, this
paper proposes CirCNN, a principled approach to represent weights and process
neural networks using block-circulant matrices. CirCNN utilizes the Fast
Fourier Transform (FFT)-based fast multiplication, simultaneously reducing the
computational complexity (both in inference and training) from O(n2) to
O(nlogn) and the storage complexity from O(n2) to O(n), with negligible
accuracy loss. Compared to other approaches, CirCNN is distinct due to its
mathematical rigor: it can converge to the same effectiveness as DNNs without
compression. The CirCNN architecture, a universal DNN inference engine that can
be implemented on various hardware/software platforms with configurable network
architecture. To demonstrate the performance and energy efficiency, we test
CirCNN in FPGA, ASIC and embedded processors. Our results show that CirCNN
architecture achieves very high energy efficiency and performance with a small
hardware footprint. Based on the FPGA implementation and ASIC synthesis
results, CirCNN achieves 6-102X energy efficiency improvements compared with
the best state-of-the-art results.
Narrative Variations in a Virtual Storyteller
Authors: Stephanie M. Lukin, Marilyn A. Walker
Date: 2017-08-29 00:00:00

Research on storytelling over the last 100 years has distinguished at least
two levels of narrative representation (1) story, or fabula; and (2) discourse,
or sujhet. We use this distinction to create Fabula Tales, a computational
framework for a virtual storyteller that can tell the same story in different
ways through the implementation of general narratological variations, such as
varying direct vs. indirect speech, character voice (style), point of view, and
focalization. A strength of our computational framework is that it is based on
very general methods for re-using existing story content, either from fables or
from personal narratives collected from blogs. We first explain how a simple
annotation tool allows naive annotators to easily create a deep representation
of fabula called a story intention graph, and show how we use this
representation to generate story tellings automatically. Then we present
results of two studies testing our narratological parameters, and showing that
different tellings affect the reader's perception of the story and characters.
Generating Sentence Planning Variations for Story Telling
Authors: Stephanie M. Lukin, Lena I. Reed, Marilyn A. Walker
Date: 2017-08-29 00:00:00
Tasks: Sentence, Story Generation, Text Generation

There has been a recent explosion in applications for dialogue interaction
ranging from direction-giving and tourist information to interactive story
systems. Yet the natural language generation (NLG) component for many of these
systems remains largely handcrafted. This limitation greatly restricts the
range of applications; it also means that it is impossible to take advantage of
recent work in expressive and statistical language generation that can
dynamically and automatically produce a large number of variations of given
content. We propose that a solution to this problem lies in new methods for
developing language generation resources. We describe the ES-Translator, a
computational language generator that has previously been applied only to
fables, and quantitatively evaluate the domain independence of the EST by
applying it to personal narratives from weblogs. We then take advantage of
recent work on language generation to create a parameterized sentence planner
for story generation that provides aggregation operations, variations in
discourse and in point of view. Finally, we present a user evaluation of
different personal narrative retellings.