pwc-archive/papers-with-abstracts huggingface dataset:
Neural Word Search in Historical Manuscript Collections
Authors: Tomas Wilkinson, Jonas Lindström, Anders Brun
Date: 2018-12-06 00:00:00

We address the problem of segmenting and retrieving word images in collections of historical manuscripts given a text query. This is commonly referred to as "word spotting". To this end, we first propose an end-to-end trainable model based on deep neural networks that we dub Ctrl-F-Net. The model simultaneously generates region proposals and embeds them into a word embedding space, wherein a search is performed. We further introduce a simplified version called Ctrl-F-Mini. It is faster with similar performance, though it is limited to more easily segmented manuscripts. We evaluate both models on common benchmark datasets and surpass the previous state of the art. Finally, in collaboration with historians, we employ the Ctrl-F-Net to search within a large manuscript collection of over 100 thousand pages, written across two centuries. With only 11 training pages, we enable large scale data collection in manuscript-based historical research. This results in a speed up of data collection and the number of manuscripts processed by orders of magnitude. Given the time consuming manual work required to study old manuscripts in the humanities, quick and robust tools for word spotting has the potential to revolutionise domains like history, religion and language.
Embedding-reparameterization procedure for manifold-valued latent variables in generative models
Authors: Eugene Golikov, Maksim Kretov
Date: 2018-12-06 00:00:00
Tasks: Machine Translation

Conventional prior for Variational Auto-Encoder (VAE) is a Gaussian
distribution. Recent works demonstrated that choice of prior distribution
affects learning capacity of VAE models. We propose a general technique
(embedding-reparameterization procedure, or ER) for introducing arbitrary
manifold-valued variables in VAE model. We compare our technique with a
conventional VAE on a toy benchmark problem. This is work in progress.
SqueezeFit: Label-aware dimensionality reduction by semidefinite programming
Authors: Culver McWhirter, Dustin G. Mixon, Soledad Villar
Date: 2018-12-06 00:00:00
Tasks: Classification, Dimensionality Reduction, General Classification

Given labeled points in a high-dimensional vector space, we seek a
low-dimensional subspace such that projecting onto this subspace maintains some
prescribed distance between points of differing labels. Intended applications
include compressive classification. Taking inspiration from large margin
nearest neighbor classification, this paper introduces a semidefinite
relaxation of this problem. Unlike its predecessors, this relaxation is
amenable to theoretical analysis, allowing us to provably recover a planted
projection operator from the data.
Knockoff Nets: Stealing Functionality of Black-Box Models
Authors: Tribhuvanesh Orekondy, Bernt Schiele, Mario Fritz
Date: 2018-12-06 00:00:00
Proceeding: CVPR 2019 6
Tasks: Reinforcement Learning

Machine Learning (ML) models are increasingly deployed in the wild to perform
a wide range of tasks. In this work, we ask to what extent can an adversary
steal functionality of such "victim" models based solely on blackbox
interactions: image in, predictions out. In contrast to prior work, we present
an adversary lacking knowledge of train/test data used by the model, its
internals, and semantics over model outputs. We formulate model functionality
stealing as a two-step approach: (i) querying a set of input images to the
blackbox model to obtain predictions; and (ii) training a "knockoff" with
queried image-prediction pairs. We make multiple remarkable observations: (a)
querying random images from a different distribution than that of the blackbox
training data results in a well-performing knockoff; (b) this is possible even
when the knockoff is represented using a different architecture; and (c) our
reinforcement learning approach additionally improves query sample efficiency
in certain settings and provides performance gains. We validate model
functionality stealing on a range of datasets and tasks, as well as on a
popular image analysis API where we create a reasonable knockoff for as little
as $30.
Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distance
Authors: Taylor Denouden, Rick Salay, Krzysztof Czarnecki, Vahdat Abdelzad, Buu Phan, Sachin Vernekar
Date: 2018-12-06 00:00:00
Tasks: Out-of-Distribution Detection

There is an increasingly apparent need for validating the classifications
made by deep learning systems in safety-critical applications like autonomous
vehicle systems. A number of recent papers have proposed methods for detecting
anomalous image data that appear different from known inlier data samples,
including reconstruction-based autoencoders. Autoencoders optimize the
compression of input data to a latent space of a dimensionality smaller than
the original input and attempt to accurately reconstruct the input using that
compressed representation. Since the latent vector is optimized to capture the
salient features from the inlier class only, it is commonly assumed that images
of objects from outside of the training class cannot effectively be compressed
and reconstructed. Some thus consider reconstruction error as a kind of novelty
measure. Here we suggest that reconstruction-based approaches fail to capture
particular anomalies that lie far from known inlier samples in latent space but
near the latent dimension manifold defined by the parameters of the model. We
propose incorporating the Mahalanobis distance in latent space to better
capture these out-of-distribution samples and our results show that this method
often improves performance over the baseline approach.
On Uncensored Mean First-Passage-Time Performance Experiments with Multiwalk in $\mathbb{R}^p$: a New Stochastic Optimization Algorithm
Authors: Franc Brglez
Date: 2018-12-06 00:00:00
Tasks: Stochastic Optimization

A rigorous empirical comparison of two stochastic solvers is important when
one of the solvers is a prototype of a new algorithm such as multiwalk (MWA).
When searching for global minima in $\mathbb{R}^p$, the key data structures of
MWA include: $p$ rulers with each ruler assigned $m$ marks and a set of $p$
neighborhood matrices of size up to $m(m-2)$, where each entry represents
absolute values of pairwise differences between $m$ marks. Before taking the
next step, a controller links the tableau of neighborhood matrices and computes
new and improved positions for each of the $m$ marks. The number of columns in
each neighborhood matrix is denoted as the neighborhood radius $r_n \le m-2$.
Any variant of the DEA (differential evolution algorithm) has an effective
population neighborhood of radius not larger than 1. Uncensored
first-passage-time performance experiments that vary the neighborhood radius of
a MW-solver can thus be readily compared to existing variants of DE-solvers.
The paper considers seven test cases of increasing complexity and demonstrates,
under uncensored first-passage-time performance experiments: (1) significant
variability in convergence rate for seven DE-based solver configurations, and
(2) consistent, monotonic, and significantly faster rate of convergence for the
MW-solver prototype as we increase the neighborhood radius from 4 to its
maximum value.
On Configurable Defense against Adversarial Example Attacks
Authors: Bo Luo, Min Li, Yu Li, Qiang Xu
Date: 2018-12-06 00:00:00

Machine learning systems based on deep neural networks (DNNs) have gained
mainstream adoption in many applications. Recently, however, DNNs are shown to
be vulnerable to adversarial example attacks with slight perturbations on the
inputs. Existing defense mechanisms against such attacks try to improve the
overall robustness of the system, but they do not differentiate different
targeted attacks even though the corresponding impacts may vary significantly.
To tackle this problem, we propose a novel configurable defense mechanism in
this work, wherein we are able to flexibly tune the robustness of the system
against different targeted attacks to satisfy application requirements. This is
achieved by refining the DNN loss function with an attack sensitive matrix to
represent the impacts of different targeted attacks. Experimental results on
CIFAR-10 and GTSRB data sets demonstrate the efficacy of the proposed solution.
A Technical Survey on Statistical Modelling and Design Methods for Crowdsourcing Quality Control
Authors: Yuan Jin, Mark Carman, Ye Zhu, Yong Xiang
Date: 2018-12-05 00:00:00
Tasks: BIG-bench Machine Learning

Online crowdsourcing provides a scalable and inexpensive means to collect
knowledge (e.g. labels) about various types of data items (e.g. text, audio,
video). However, it is also known to result in large variance in the quality of
recorded responses which often cannot be directly used for training machine
learning systems. To resolve this issue, a lot of work has been conducted to
control the response quality such that low-quality responses cannot adversely
affect the performance of the machine learning systems. Such work is referred
to as the quality control for crowdsourcing. Past quality control research can
be divided into two major branches: quality control mechanism design and
statistical models. The first branch focuses on designing measures, thresholds,
interfaces and workflows for payment, gamification, question assignment and
other mechanisms that influence workers' behaviour. The second branch focuses
on developing statistical models to perform effective aggregation of responses
to infer correct responses. The two branches are connected as statistical
models (i) provide parameter estimates to support the measure and threshold
calculation, and (ii) encode modelling assumptions used to derive (theoretical)
performance guarantees for the mechanisms. There are surveys regarding each
branch but they lack technical details about the other branch. Our survey is
the first to bridge the two branches by providing technical details on how they
work together under frameworks that systematically unify crowdsourcing aspects
modelled by both of them to determine the response quality. We are also the
first to provide taxonomies of quality control papers based on the proposed
frameworks. Finally, we specify the current limitations and the corresponding
future directions for the quality control research.
Learning to Design Circuits
Authors: Hanrui Wang, Jiacheng Yang, Hae-Seung Lee, Song Han
Date: 2018-12-05 00:00:00
Tasks: Bayesian Optimization, Reinforcement Learning

Analog IC design relies on human experts to search for parameters that satisfy circuit specifications with their experience and intuitions, which is highly labor intensive, time consuming and suboptimal. Machine learning is a promising tool to automate this process. However, supervised learning is difficult for this task due to the low availability of training data: 1) Circuit simulation is slow, thus generating large-scale dataset is time-consuming; 2) Most circuit designs are propitiatory IPs within individual IC companies, making it expensive to collect large-scale datasets. We propose Learning to Design Circuits (L2DC) to leverage reinforcement learning that learns to efficiently generate new circuits data and to optimize circuits. We fix the schematic, and optimize the parameters of the transistors automatically by training an RL agent with no prior knowledge about optimizing circuits. After iteratively getting observations, generating a new set of transistor parameters, getting a reward, and adjusting the model, L2DC is able to optimize circuits. We evaluate L2DC on two transimpedance amplifiers. Trained for a day, our RL agent can achieve comparable or better performance than human experts trained for a quarter. It first learns to meet hard-constraints (eg. gain, bandwidth), and then learns to optimize good-to-have targets (eg. area, power). Compared with grid search-aided human design, L2DC can achieve $\mathbf{250}\boldsymbol{\times}$ higher sample efficiency with comparable performance. Under the same runtime constraint, the performance of L2DC is also better than Bayesian Optimization.
Deep Learning at Scale for the Construction of Galaxy Catalogs in the Dark Energy Survey
Authors: Asad Khan, E. A. Huerta, Sibo Wang, Robert Gruendl, Elise Jennings, Huihuo Zheng
Date: 2018-12-05 00:00:00
Tasks: Clustering, Survey

The scale of ongoing and future electromagnetic surveys pose formidable challenges to classify astronomical objects. Pioneering efforts on this front include citizen science campaigns adopted by the Sloan Digital Sky Survey (SDSS). SDSS datasets have been recently used to train neural network models to classify galaxies in the Dark Energy Survey (DES) that overlap the footprint of both surveys. Herein, we demonstrate that knowledge from deep learning algorithms, pre-trained with real-object images, can be transferred to classify galaxies that overlap both SDSS and DES surveys, achieving state-of-the-art accuracy $\gtrsim99.6\%$. We demonstrate that this process can be completed within just eight minutes using distributed training. While this represents a significant step towards the classification of DES galaxies that overlap previous surveys, we need to initiate the characterization of unlabelled DES galaxies in new regions of parameter space. To accelerate this program, we use our neural network classifier to label over ten thousand unlabelled DES galaxies, which do not overlap previous surveys. Furthermore, we use our neural network model as a feature extractor for unsupervised clustering and find that unlabeled DES images can be grouped together in two distinct galaxy classes based on their morphology, which provides a heuristic check that the learning is successfully transferred to the classification of unlabelled DES images. We conclude by showing that these newly labeled datasets can be combined with unsupervised recursive training to create large-scale DES galaxy catalogs in preparation for the Large Synoptic Survey Telescope era.
Attending to Mathematical Language with Transformers
Authors: Artit Wangperawong
Date: 2018-12-05 00:00:00
Tasks: Decoder

Mathematical expressions were generated, evaluated and used to train neural network models based on the transformer architecture. The expressions and their targets were analyzed as a character-level sequence transduction task in which the encoder and decoder are built on attention mechanisms. Three models were trained to understand and evaluate symbolic variables and expressions in mathematics: (1) the self-attentive and feed-forward transformer without recurrence or convolution, (2) the universal transformer with recurrence, and (3) the adaptive universal transformer with recurrence and adaptive computation time. The models respectively achieved test accuracies as high as 76.1%, 78.8% and 84.9% in evaluating the expressions to match the target values. For the cases inferred incorrectly, the results differed from the targets by only one or two characters. The models notably learned to add, subtract and multiply both positive and negative decimal numbers of variable digits assigned to symbolic variables.
Multiple Manifold Clustering Using Curvature Constrained Path
Authors: Amir Babaeian
Date: 2018-12-04 00:00:00
Tasks: Clustering

The problem of multiple surface clustering is a challenging task,
particularly when the surfaces intersect. Available methods such as Isomap fail
to capture the true shape of the surface nearby the intersection and result in
incorrect clustering. The Isomap algorithm uses the shortest path between
points. The main draw back of the shortest path algorithm is due to the lack of
curvature constrained where causes to have a path between points on different
surfaces. In this paper, we tackle this problem by imposing a curvature
constraint to the shortest path algorithm used in Isomap. The algorithm chooses
several landmark nodes at random and then checks whether there is a curvature
constrained path between each landmark node and every other node in the
neighbourhood graph. We build a binary feature vector for each point where each
entry represents the connectivity of that point to a particular landmark. Then
the binary feature vectors could be used as an input of conventional clustering
algorithm such as hierarchical clustering. We apply our method to simulated and
some real datasets and show, it performs comparably to the best methods such as
K-manifold and spectral multi-manifold clustering.
Utilizing Imbalanced Data and Classification Cost Matrix to Predict Movie Preferences
Authors: Haifeng Wang
Date: 2018-12-04 00:00:00
Tasks: Classification, General Classification, Marketing, Movie Genre Recommendation System, Recommendation Systems

In this paper, we propose a movie genre recommendation system based on
imbalanced survey data and unequal classification costs for small and
medium-sized enterprises (SMEs) who need a data-based and analytical approach
to stock favored movies and target marketing to young people. The dataset
maintains a detailed personal profile as predictors including demographic,
behavioral and preferences information for each user as well as imbalanced
genre preferences. These predictors do not include the information such as
actors or directors. The paper applies Gentle boost, Adaboost and Bagged tree
ensembles as well as SVM machine learning algorithms to learn classification
from one thousand observations and predict movie genre preferences with
adjusted classification costs. The proposed recommendation system also selects
important predictors to avoid overfitting and to shorten training time. This
paper compares the test error among the above-mentioned algorithms that are
used to recommend different movie genres. The prediction power is also
indicated in a comparison of precision and recall with other state-of-the-art
recommendation systems. The proposed movie genre recommendation system solves
problems such as small dataset, imbalanced response, and unequal classification
costs.
q-LMF: Quantum Calculus-based Least Mean Fourth Algorithm
Authors: Alishba Sadiq, Muhammad Usman, Shujaat Khan, Imran Naseem, Muhammad Moinuddin, Ubaid M. Al-Saggaf
Date: 2018-12-04 00:00:00

Channel estimation is an essential part of modern communication systems as it
enhances the overall performance of the system. In recent past a variety of
adaptive learning methods have been designed to enhance the robustness and
convergence speed of the learning process. However, the need for an optimal
technique is still there. Herein, for non-Gaussian noisy environment we propose
a new class of stochastic gradient algorithm for channel identification. The
proposed $q$-least mean fourth ($q$-LMF) is an extension of least mean fourth
(LMF) algorithm and it is based on the $q$-calculus which is also known as
Jackson derivative. The proposed algorithm utilizes a novel concept of
error-correlation energy and normalization of signal to ensure high convergence
rate, better stability and low steady-state error. Contrary to the conventional
LMF, the proposed method has more freedom for large step-sizes. Extensive
experiments show significant gain in the performance of the proposed $q$-LMF
algorithm in comparison to the contemporary techniques.
Voice Disorder Detection Using Long Short Term Memory (LSTM) Model
Authors: Vibhuti Gupta
Date: 2018-12-04 00:00:00
Tasks: Specificity

Automated detection of voice disorders with computational methods is a recent
research area in the medical domain since it requires a rigorous endoscopy for
the accurate diagnosis. Efficient screening methods are required for the
diagnosis of voice disorders so as to provide timely medical facilities in
minimal resources. Detecting Voice disorder using computational methods is a
challenging problem since audio data is continuous due to which extracting
relevant features and applying machine learning is hard and unreliable. This
paper proposes a Long short term memory model (LSTM) to detect pathological
voice disorders and evaluates its performance in a real 400 testing samples
without any labels. Different feature extraction methods are used to provide
the best set of features before applying LSTM model for classification. The
paper describes the approach and experiments that show promising results with
22% sensitivity, 97% specificity and 56% unweighted average recall.
Selected Qualitative Spatio-temporal Calculi Developed for Constraint Reasoning: A Review
Authors: Debasis Mitra
Date: 2018-12-03 00:00:00
Tasks: Topological Data Analysis

In this article a few of the qualitative spatio-temporal knowledge
representation techniques developed by the constraint reasoning community
within artificial intelligence are reviewed. The objective is to provide a
broad exposure to any other interested group who may utilize these
representations. The author has a particular interest in applying these calculi
(in a broad sense) in topological data analysis, as these schemes are highly
qualitative in nature.
Conscious enactive computation
Authors: Daniel Estrada
Date: 2018-12-03 00:00:00

This paper looks at recent debates in the enactivist literature on
computation and consciousness in order to assess major obstacles to building
artificial conscious agents. We consider a proposal from Villalobos and
Dewhurst (2018) for enactive computation on the basis of organizational
closure. We attempt to improve the argument by reflecting on the closed paths
through state space taken by finite state automata. This motivates a defense
against Clark's recent criticisms of "extended consciousness", and perhaps a
new perspective on living with machines.
AsyncQVI: Asynchronous-Parallel Q-Value Iteration for Discounted Markov Decision Processes with Near-Optimal Sample Complexity
Authors: Yibo Zeng, Fei Feng, Wotao Yin
Date: 2018-12-03 00:00:00

In this paper, we propose AsyncQVI, an asynchronous-parallel Q-value iteration for discounted Markov decision processes whose transition and reward can only be sampled through a generative model. Given such a problem with $|\mathcal{S}|$ states, $|\mathcal{A}|$ actions, and a discounted factor $\gamma\in(0,1)$, AsyncQVI uses memory of size $\mathcal{O}(|\mathcal{S}|)$ and returns an $\varepsilon$-optimal policy with probability at least $1-\delta$ using $$\tilde{\mathcal{O}}\big(\frac{|\mathcal{S}||\mathcal{A}|}{(1-\gamma)^5\varepsilon^2}\log(\frac{1}{\delta})\big)$$ samples. AsyncQVI is also the first asynchronous-parallel algorithm for discounted Markov decision processes that has a sample complexity, which nearly matches the theoretical lower bound. The relatively low memory footprint and parallel ability make AsyncQVI suitable for large-scale applications. In numerical tests, we compare AsyncQVI with four sample-based value iteration methods. The results show that our algorithm is highly efficient and achieves linear parallel speedup.