pwc-archive/papers-with-abstracts huggingface dataset:
Meta-Learning: A Survey
Authors: Joaquin Vanschoren
Date: 2018-10-08 00:00:00
Tasks: BIG-bench Machine Learning, Meta-Learning, Survey

Meta-learning, or learning to learn, is the science of systematically
observing how different machine learning approaches perform on a wide range of
learning tasks, and then learning from this experience, or meta-data, to learn
new tasks much faster than otherwise possible. Not only does this dramatically
speed up and improve the design of machine learning pipelines or neural
architectures, it also allows us to replace hand-engineered algorithms with
novel approaches learned in a data-driven way. In this chapter, we provide an
overview of the state of the art in this fascinating and continuously evolving
field.
Stein Neural Sampler
Authors: Tianyang Hu, Zixiang Chen, Hanxi Sun, Jincheng Bai, Mao Ye, Guang Cheng
Date: 2018-10-08 00:00:00

We propose two novel samplers to generate high-quality samples from a given (un-normalized) probability density. Motivated by the success of generative adversarial networks, we construct our samplers using deep neural networks that transform a reference distribution to the target distribution. Training schemes are developed to minimize two variations of the Stein discrepancy, which is designed to work with un-normalized densities. Once trained, our samplers are able to generate samples instantaneously. We show that the proposed methods are theoretically sound and experience fewer convergence issues compared with traditional sampling approaches according to our empirical studies.
An AMR Aligner Tuned by Transition-based Parser
Authors: Yijia Liu, Wanxiang Che, Bo Zheng, Bing Qin, Ting Liu
Date: 2018-10-08 00:00:00
Proceeding: EMNLP 2018 10
Tasks: AMR Parsing, POS, Sentence

In this paper, we propose a new rich resource enhanced AMR aligner which
produces multiple alignments and a new transition system for AMR parsing along
with its oracle parser. Our aligner is further tuned by our oracle parser via
picking the alignment that leads to the highest-scored achievable AMR graph.
Experimental results show that our aligner outperforms the rule-based aligner
in previous work by achieving higher alignment F1 score and consistently
improving two open-sourced AMR parsers. Based on our aligner and transition
system, we develop a transition-based AMR parser that parses a sentence into
its AMR graph directly. An ensemble of our parsers with only words and POS tags
as input leads to 68.4 Smatch F1 score.
Combinatorial Attacks on Binarized Neural Networks
Authors: Elias B. Khalil, Amrita Gupta, Bistra Dilkina
Date: 2018-10-08 00:00:00
Proceeding: ICLR 2019 5
Tasks: Computational Efficiency

Binarized Neural Networks (BNNs) have recently attracted significant interest
due to their computational efficiency. Concurrently, it has been shown that
neural networks may be overly sensitive to "attacks" - tiny adversarial changes
in the input - which may be detrimental to their use in safety-critical
domains. Designing attack algorithms that effectively fool trained models is a
key step towards learning robust neural networks. The discrete,
non-differentiable nature of BNNs, which distinguishes them from their
full-precision counterparts, poses a challenge to gradient-based attacks. In
this work, we study the problem of attacking a BNN through the lens of
combinatorial and integer optimization. We propose a Mixed Integer Linear
Programming (MILP) formulation of the problem. While exact and flexible, the
MILP quickly becomes intractable as the network and perturbation space grow. To
address this issue, we propose IProp, a decomposition-based algorithm that
solves a sequence of much smaller MILP problems. Experimentally, we evaluate
both proposed methods against the standard gradient-based attack (FGSM) on
MNIST and Fashion-MNIST, and show that IProp performs favorably compared to
FGSM, while scaling beyond the limits of the MILP.
Effective Parallelisation for Machine Learning
Authors: Michael Kamp, Mario Boley, Olana Missura, Thomas Gärtner
Date: 2018-10-08 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: BIG-bench Machine Learning, Open-Ended Question Answering

We present a novel parallelisation scheme that simplifies the adaptation of
learning algorithms to growing amounts of data as well as growing needs for
accurate and confident predictions in critical applications. In contrast to
other parallelisation techniques, it can be applied to a broad class of
learning algorithms without further mathematical derivations and without
writing dedicated code, while at the same time maintaining theoretical
performance guarantees. Moreover, our parallelisation scheme is able to reduce
the runtime of many learning algorithms to polylogarithmic time on
quasi-polynomially many processing units. This is a significant step towards a
general answer to an open question on the efficient parallelisation of machine
learning algorithms in the sense of Nick's Class (NC). The cost of this
parallelisation is in the form of a larger sample complexity. Our empirical
study confirms the potential of our parallelisation scheme with fixed numbers
of processors and instances in realistic application scenarios.
CHOPT : Automated Hyperparameter Optimization Framework for Cloud-Based Machine Learning Platforms
Authors: Jinwoong Kim, Minkyu Kim, Heungseok Park, Ernar Kusdavletov, Dongjun Lee, Adrian Kim, Ji-Hoon Kim, Jung-Woo Ha, Nako Sung
Date: 2018-10-08 00:00:00
Tasks: BIG-bench Machine Learning, Hyperparameter Optimization, Question Answering

Many hyperparameter optimization (HyperOpt) methods assume restricted
computing resources and mainly focus on enhancing performance. Here we propose
a novel cloud-based HyperOpt (CHOPT) framework which can efficiently utilize
shared computing resources while supporting various HyperOpt algorithms. We
incorporate convenient web-based user interfaces, visualization, and analysis
tools, enabling users to easily control optimization procedures and build up
valuable insights with an iterative analysis procedure. Furthermore, our
framework can be incorporated with any cloud platform, thus complementarily
increasing the efficiency of conventional deep learning frameworks. We
demonstrate applications of CHOPT with tasks such as image recognition and
question-answering, showing that our framework can find hyperparameter
configurations competitive with previous work. We also show CHOPT is capable of
providing interesting observations through its analysing tools
Trace Quotient with Sparsity Priors for Learning Low Dimensional Image Representations
Authors: Xian Wei, Hao Shen, Martin Kleinsteuber
Date: 2018-10-08 00:00:00
Tasks: Data Visualization, Dimensionality Reduction, Representation Learning

This work studies the problem of learning appropriate low dimensional image
representations. We propose a generic algorithmic framework, which leverages
two classic representation learning paradigms, i.e., sparse representation and
the trace quotient criterion. The former is a well-known powerful tool to
identify underlying self-explanatory factors of data, while the latter is known
for disentangling underlying low dimensional discriminative factors in data.
Our developed solutions disentangle sparse representations of images by
employing the trace quotient criterion. We construct a unified cost function,
coined as the SPARse LOW dimensional representation (SparLow) function, for
jointly learning both a sparsifying dictionary and a dimensionality reduction
transformation. The SparLow function is widely applicable for developing
various algorithms in three classic machine learning scenarios, namely,
unsupervised, supervised, and semi-supervised learning. In order to develop
efficient joint learning algorithms for maximizing the SparLow function, we
deploy a framework of sparse coding with appropriate convex priors to ensure
the sparse representations to be locally differentiable. Moreover, we develop
an efficient geometric conjugate gradient algorithm to maximize the SparLow
function on its underlying Riemannian manifold. Performance of the proposed
SparLow algorithmic framework is investigated on several image processing
tasks, such as 3D data visualization, face/digit recognition, and object/scene
categorization.
NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm
Authors: Zhichao Lu, Ian Whalen, Vishnu Boddeti, Yashesh Dhebar, Kalyanmoy Deb, Erik Goodman, Wolfgang Banzhaf
Date: 2018-10-08 00:00:00
Tasks: Efficient Exploration, Neural Architecture Search

This paper introduces NSGA-Net -- an evolutionary approach for neural
architecture search (NAS). NSGA-Net is designed with three goals in mind: (1) a
procedure considering multiple and conflicting objectives, (2) an efficient
procedure balancing exploration and exploitation of the space of potential
neural network architectures, and (3) a procedure finding a diverse set of
trade-off network architectures achieved in a single run. NSGA-Net is a
population-based search algorithm that explores a space of potential neural
network architectures in three steps, namely, a population initialization step
that is based on prior-knowledge from hand-crafted architectures, an
exploration step comprising crossover and mutation of architectures, and
finally an exploitation step that utilizes the hidden useful knowledge stored
in the entire history of evaluated neural architectures in the form of a
Bayesian Network. Experimental results suggest that combining the dual
objectives of minimizing an error metric and computational complexity, as
measured by FLOPs, allows NSGA-Net to find competitive neural architectures.
Moreover, NSGA-Net achieves error rate on the CIFAR-10 dataset on par with
other state-of-the-art NAS methods while using orders of magnitude less
computational resources. These results are encouraging and shows the promise to
further use of EC methods in various deep-learning paradigms.
A Droplet Approach Based on Raptor Codes for Distributed Computing With Straggling Servers
Authors: Albin Severinson, Alexandre Graell i Amat, Eirik Rosnes, Francisco Lazaro, Gianluigi Liva
Date: 2018-10-08 00:00:00
Tasks: Distributed Computing

We propose a coded distributed computing scheme based on Raptor codes to
address the straggler problem. In particular, we consider a scheme where each
server computes intermediate values, referred to as droplets, that are either
stored locally or sent over the network. Once enough droplets are collected,
the computation can be completed. Compared to previous schemes in the
literature, our proposed scheme achieves lower computational delay when the
decoding time is taken into account.
Security Analysis of Deep Neural Networks Operating in the Presence of Cache Side-Channel Attacks
Authors: Sanghyun Hong, Michael Davinroy, Yiǧitcan Kaya, Stuart Nevans Locke, Ian Rackow, Kevin Kulda, Dana Dachman-Soled, Tudor Dumitraş
Date: 2018-10-08 00:00:00
Proceeding: ICLR 2019 5
Tasks: Transfer Learning

Recent work has introduced attacks that extract the architecture information of deep neural networks (DNN), as this knowledge enhances an adversary's capability to conduct black-box attacks against the model. This paper presents the first in-depth security analysis of DNN fingerprinting attacks that exploit cache side-channels. First, we define the threat model for these attacks: our adversary does not need the ability to query the victim model; instead, she runs a co-located process on the host machine victim's deep learning (DL) system is running and passively monitors the accesses of the target functions in the shared framework. Second, we introduce DeepRecon, an attack that reconstructs the architecture of the victim network by using the internal information extracted via Flush+Reload, a cache side-channel technique. Once the attacker observes function invocations that map directly to architecture attributes of the victim network, the attacker can reconstruct the victim's entire network architecture. In our evaluation, we demonstrate that an attacker can accurately reconstruct two complex networks (VGG19 and ResNet50) having observed only one forward propagation. Based on the extracted architecture attributes, we also demonstrate that an attacker can build a meta-model that accurately fingerprints the architecture and family of the pre-trained model in a transfer learning setting. From this meta-model, we evaluate the importance of the observed attributes in the fingerprinting process. Third, we propose and evaluate new framework-level defense techniques that obfuscate our attacker's observations. Our empirical security analysis represents a step toward understanding the DNNs' vulnerability to cache side-channel attacks.
Towards the Latent Transcriptome
Authors: Assya Trofimov, Francis Dutil, Claude Perreault, Sebastien Lemieux, Yoshua Bengio, Joseph Paul Cohen
Date: 2018-10-08 00:00:00

In this work we propose a method to compute continuous embeddings for kmers
from raw RNA-seq data, without the need for alignment to a reference genome.
The approach uses an RNN to transform kmers of the RNA-seq reads into a 2
dimensional representation that is used to predict abundance of each kmer. We
report that our model captures information of both DNA sequence similarity as
well as DNA sequence abundance in the embedding latent space, that we call the
Latent Transcriptome. We confirm the quality of these vectors by comparing them
to known gene sub-structures and report that the latent space recovers exon
information from raw RNA-Seq data from acute myeloid leukemia patients.
Furthermore we show that this latent space allows the detection of genomic
abnormalities such as translocations as well as patient-specific mutations,
making this representation space both useful for visualization as well as
analysis.
State of the Art Optical Character Recognition of 19th Century Fraktur Scripts using Open Source Engines
Authors: Christian Reul, Uwe Springmann, Christoph Wick, Frank Puppe
Date: 2018-10-08 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR)

In this paper we evaluate Optical Character Recognition (OCR) of 19th century
Fraktur scripts without book-specific training using mixed models, i.e. models
trained to recognize a variety of fonts and typesets from previously unseen
sources. We describe the training process leading to strong mixed OCR models
and compare them to freely available models of the popular open source engines
OCRopus and Tesseract as well as the commercial state of the art system ABBYY.
For evaluation, we use a varied collection of unseen data from books, journals,
and a dictionary from the 19th century. The experiments show that training
mixed models with real data is superior to training with synthetic data and
that the novel OCR engine Calamari outperforms the other engines considerably,
on average reducing ABBYYs character error rate (CER) by over 70%, resulting in
an average CER below 1%.
Cross Script Hindi English NER Corpus from Wikipedia
Authors: Mohd Zeeshan Ansari, Tanvir Ahmad, Md Arshad Ali
Date: 2018-10-08 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

The text generated on social media platforms is essentially a mixed lingual
text. The mixing of language in any form produces considerable amount of
difficulty in language processing systems. Moreover, the advancements in
language processing research depends upon the availability of standard corpora.
The development of mixed lingual Indian Named Entity Recognition (NER) systems
are facing obstacles due to unavailability of the standard evaluation corpora.
Such corpora may be of mixed lingual nature in which text is written using
multiple languages predominantly using a single script only. The motivation of
our work is to emphasize the automatic generation such kind of corpora in order
to encourage mixed lingual Indian NER. The paper presents the preparation of a
Cross Script Hindi-English Corpora from Wikipedia category pages. The corpora
is successfully annotated using standard CoNLL-2003 categories of PER, LOC,
ORG, and MISC. Its evaluation is carried out on a variety of machine learning
algorithms and favorable results are achieved.
MRI Super-Resolution using Multi-Channel Total Variation
Authors: Mikael Brudfors, Yael Balbastre, Parashkev Nachev, John Ashburner
Date: 2018-10-08 00:00:00
Tasks: Brain Segmentation, Super-Resolution

This paper presents a generative model for super-resolution in routine clinical magnetic resonance images (MRI), of arbitrary orientation and contrast. The model recasts the recovery of high resolution images as an inverse problem, in which a forward model simulates the slice-select profile of the MR scanner. The paper introduces a prior based on multi-channel total variation for MRI super-resolution. Bias-variance trade-off is handled by estimating hyper-parameters from the low resolution input scans. The model was validated on a large database of brain images. The validation showed that the model can improve brain segmentation, that it can recover anatomical information between images of different MR contrasts, and that it generalises well to the large variability present in MR images of different subjects. The implementation is freely available at https://github.com/brudfors/spm_superres
Unique Metric for Health Analysis with Optimization of Clustering Activity and Cross Comparison of Results from Different Approach
Authors: Kumarjit Pathak, Jitin Kapila
Date: 2018-10-08 00:00:00
Tasks: Clustering, Descriptive, Philosophy

In machine learning and data mining, Cluster analysis is one of the most
widely used unsupervised learning technique. Philosophy of this algorithm is to
find similar data items and group them together based on any distance function
in multidimensional space. These methods are suitable for finding groups of
data that behave in a coherent fashion. The perspective may vary for clustering
i.e. the way we want to find similarity, some methods are based on distance
such as K-Means technique and some are probability based, like GMM.
Understanding prominent segment of data is always challenging as multidimension
space does not allow us to have a look and feel of the distance or any visual
context on the health of the clustering.
  While explaining data using clusters, the major problem is to tell how many
cluster are good enough to explain the data. Generally basic descriptive
statistics are used to estimate cluster behaviour like scree plot, dendrogram
etc. We propose a novel method to understand the cluster behaviour which can be
used not only to find right number of clusters but can also be used to access
the difference of health between different clustering methods on same data. Our
technique would also help to also eliminate the noisy variables and optimize
the clustering result.
  keywords - Clustering, Metric, K-means, hierarchical clustering, silhoutte,
clustering index, measures
POLO: a POLicy-based Optimization library
Authors: Arda Aytekin, Martin Biel, Mikael Johansson
Date: 2018-10-08 00:00:00

We present POLO --- a C++ library for large-scale parallel optimization
research that emphasizes ease-of-use, flexibility and efficiency in algorithm
design. It uses multiple inheritance and template programming to decompose
algorithms into essential policies and facilitate code reuse. With its clear
separation between algorithm and execution policies, it provides researchers
with a simple and powerful platform for prototyping ideas, evaluating them on
different parallel computing architectures and hardware platforms, and
generating compact and efficient production code. A C-API is included for
customization and data loading in high-level languages. POLO enables users to
move seamlessly from serial to multi-threaded shared-memory and multi-node
distributed-memory executors. We demonstrate how POLO allows users to implement
state-of-the-art asynchronous parallel optimization algorithms in just a few
lines of code and report experiment results from shared and distributed-memory
computing architectures. We provide both POLO and POLO.jl, a wrapper around
POLO written in the Julia language, at https://github.com/pologrp under the
permissive MIT license.
Dense Multimodal Fusion for Hierarchically Joint Representation
Authors: Di Hu, Feiping Nie, Xuelong. Li
Date: 2018-10-08 00:00:00
Tasks: Cross-Modal Retrieval, Retrieval, speech-recognition, Speech Recognition

Multiple modalities can provide more valuable information than single one by
describing the same contents in various ways. Hence, it is highly expected to
learn effective joint representation by fusing the features of different
modalities. However, previous methods mainly focus on fusing the shallow
features or high-level representations generated by unimodal deep networks,
which only capture part of the hierarchical correlations across modalities. In
this paper, we propose to densely integrate the representations by greedily
stacking multiple shared layers between different modality-specific networks,
which is named as Dense Multimodal Fusion (DMF). The joint representations in
different shared layers can capture the correlations in different levels, and
the connection between shared layers also provides an efficient way to learn
the dependence among hierarchical correlations. These two properties jointly
contribute to the multiple learning paths in DMF, which results in faster
convergence, lower training loss, and better performance. We evaluate our model
on three typical multimodal learning tasks, including audiovisual speech
recognition, cross-modal retrieval, and multimodal classification. The
noticeable performance in the experiments demonstrates that our model can learn
more effective joint representation.
Robust 6D Object Pose Estimation in Cluttered Scenes using Semantic Segmentation and Pose Regression Networks
Authors: Arul Selvam Periyasamy, Max Schwarz, Sven Behnke
Date: 2018-10-08 00:00:00
Tasks: 6D Pose Estimation using RGB, Object, Pose Estimation, regression, Segmentation, Semantic Segmentation, Synthetic Data Generation

Object pose estimation is a crucial prerequisite for robots to perform
autonomous manipulation in clutter. Real-world bin-picking settings such as
warehouses present additional challenges, e.g., new objects are added
constantly. Most of the existing object pose estimation methods assume that 3D
models of the objects is available beforehand. We present a pipeline that
requires minimal human intervention and circumvents the reliance on the
availability of 3D models by a fast data acquisition method and a synthetic
data generation procedure. This work builds on previous work on semantic
segmentation of cluttered bin-picking scenes to isolate individual objects in
clutter. An additional network is trained on synthetic scenes to estimate
object poses from a cropped object-centered encoding extracted from the
segmentation results. The proposed method is evaluated on a synthetic
validation dataset and cluttered real-world scenes.
Deep LDA Hashing
Authors: Di Hu, Feiping Nie, Xuelong. Li
Date: 2018-10-08 00:00:00

The conventional supervised hashing methods based on classification do not
entirely meet the requirements of hashing technique, but Linear Discriminant
Analysis (LDA) does. In this paper, we propose to perform a revised LDA
objective over deep networks to learn efficient hashing codes in a truly
end-to-end fashion. However, the complicated eigenvalue decomposition within
each mini-batch in every epoch has to be faced with when simply optimizing the
deep network w.r.t. the LDA objective. In this work, the revised LDA objective
is transformed into a simple least square problem, which naturally overcomes
the intractable problems and can be easily solved by the off-the-shelf
optimizer. Such deep extension can also overcome the weakness of LDA Hashing in
the limited linear projection and feature learning. Amounts of experiments are
conducted on three benchmark datasets. The proposed Deep LDA Hashing shows
nearly 70 points improvement over the conventional one on the CIFAR-10 dataset.
It also beats several state-of-the-art methods on various metrics.