pwc-archive/papers-with-abstracts huggingface dataset:
Invariant Information Clustering for Unsupervised Image Classification and Segmentation
Authors: Xu Ji, João F. Henriques, Andrea Vedaldi
Date: 2018-07-17 00:00:00
Proceeding: ICCV 2019 10
Tasks: Clustering, General Classification, image-classification, Image Classification, Image Clustering, Semantic Segmentation, Unsupervised Image Classification, Unsupervised MNIST, Unsupervised Semantic Segmentation

We present a novel clustering objective that learns a neural network classifier from scratch, given only unlabelled data samples. The model discovers clusters that accurately match semantic classes, achieving state-of-the-art results in eight unsupervised clustering benchmarks spanning image classification and segmentation. These include STL10, an unsupervised variant of ImageNet, and CIFAR10, where we significantly beat the accuracy of our closest competitors by 6.6 and 9.5 absolute percentage points respectively. The method is not specialised to computer vision and operates on any paired dataset samples; in our experiments we use random transforms to obtain a pair from each image. The trained network directly outputs semantic labels, rather than high dimensional representations that need external processing to be usable for semantic clustering. The objective is simply to maximise mutual information between the class assignments of each pair. It is easy to implement and rigorously grounded in information theory, meaning we effortlessly avoid degenerate solutions that other clustering methods are susceptible to. In addition to the fully unsupervised mode, we also test two semi-supervised settings. The first achieves 88.8% accuracy on STL10 classification, setting a new global state-of-the-art over all existing methods (whether supervised, semi-supervised or unsupervised). The second shows robustness to 90% reductions in label coverage, of relevance to applications that wish to make use of small amounts of labels. github.com/xu-ji/IIC
Item Recommendation with Variational Autoencoders and Heterogenous Priors
Authors: Giannis Karamanolakis, Kevin Raji Cherian, Ananth Ravi Narayan, Jie Yuan, Da Tang, Tony Jebara
Date: 2018-07-17 00:00:00
Tasks: Collaborative Filtering

In recent years, Variational Autoencoders (VAEs) have been shown to be highly
effective in both standard collaborative filtering applications and extensions
such as incorporation of implicit feedback. We extend VAEs to collaborative
filtering with side information, for instance when ratings are combined with
explicit text feedback from the user. Instead of using a user-agnostic standard
Gaussian prior, we incorporate user-dependent priors in the latent VAE space to
encode users' preferences as functions of the review text. Taking into account
both the rating and the text information to represent users in this multimodal
latent space is promising to improve recommendation quality. Our proposed model
is shown to outperform the existing VAE models for collaborative filtering (up
to 29.41% relative improvement in ranking metric) along with other baselines
that incorporate both user ratings and text for item recommendation.
Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions
Authors: Tim Sainburg, Marvin Thielk, Brad Theilman, Benjamin Migliori, Timothy Gentner
Date: 2018-07-17 00:00:00
Tasks: Generative Adversarial Network

We present a neural network architecture based upon the Autoencoder (AE) and
Generative Adversarial Network (GAN) that promotes a convex latent distribution
by training adversarially on latent space interpolations. By using an AE as
both the generator and discriminator of a GAN, we pass a pixel-wise error
function across the discriminator, yielding an AE which produces non-blurry
samples that match both high- and low-level features of the original images.
Interpolations between images in this space remain within the latent-space
distribution of real images as trained by the discriminator, and therfore
preserve realistic resemblances to the network inputs. Code available at
https://github.com/timsainb/GAIA
A Framework for Moment Invariants
Authors: Omar Tahri
Date: 2018-07-17 00:00:00
Tasks: Object

For more than half a century, moments have attracted lot ot interest in the
pattern recognition community.The moments of a distribution (an object) provide
several of its characteristics as center of gravity, orientation, disparity,
volume. Moments can be used to define invariant characteristics to some
transformations that an object can undergo, commonly called moment invariants.
This work provides a simple and systematic formalism to compute geometric
moment invariants in n-dimensional space.
Developing a Portable Natural Language Processing Based Phenotyping System
Authors: Himanshu Sharma, Chengsheng Mao, Yizhen Zhang, Haleh Vatani, Liang Yao, Yizhen Zhong, Luke Rasmussen, Guoqian Jiang, Jyotishman Pathak, Yuan Luo
Date: 2018-07-17 00:00:00
Tasks: BIG-bench Machine Learning

This paper presents a portable phenotyping system that is capable of
integrating both rule-based and statistical machine learning based approaches.
Our system utilizes UMLS to extract clinically relevant features from the
unstructured text and then facilitates portability across different
institutions and data systems by incorporating OHDSI's OMOP Common Data Model
(CDM) to standardize necessary data elements. Our system can also store the key
components of rule-based systems (e.g., regular expression matches) in the
format of OMOP CDM, thus enabling the reuse, adaptation and extension of many
existing rule-based clinical NLP systems. We experimented with our system on
the corpus from i2b2's Obesity Challenge as a pilot study. Our system
facilitates portable phenotyping of obesity and its 15 comorbidities based on
the unstructured patient discharge summaries, while achieving a performance
that often ranked among the top 10 of the challenge participants. This
standardization enables a consistent application of numerous rule-based and
machine learning based classification techniques downstream.
Expressive power of outer product manifolds on feed-forward neural networks
Authors: Bálint Daróczy, Rita Aleksziev, András Benczúr
Date: 2018-07-17 00:00:00

Hierarchical neural networks are exponentially more efficient than their
corresponding "shallow" counterpart with the same expressive power, but involve
huge number of parameters and require tedious amounts of training. Our main
idea is to mathematically understand and describe the hierarchical structure of
feedforward neural networks by reparametrization invariant Riemannian metrics.
By computing or approximating the tangent subspace, we better utilize the
original network via sparse representations that enables switching to shallow
networks after a very early training stage. Our experiments show that the
proposed approximation of the metric improves and sometimes even surpasses the
achievable performance of the original network significantly even after a few
epochs of training the original feedforward network.
Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning
Authors: Hao Yu, Sen yang, Shenghuo Zhu
Date: 2018-07-17 00:00:00

In distributed training of deep neural networks, parallel mini-batch SGD is
widely used to speed up the training process by using multiple workers. It uses
multiple workers to sample local stochastic gradient in parallel, aggregates
all gradients in a single server to obtain the average, and update each
worker's local model using a SGD update with the averaged gradient. Ideally,
parallel mini-batch SGD can achieve a linear speed-up of the training time
(with respect to the number of workers) compared with SGD over a single worker.
However, such linear scalability in practice is significantly limited by the
growing demand for gradient communication as more workers are involved. Model
averaging, which periodically averages individual models trained over parallel
workers, is another common practice used for distributed training of deep
neural networks since (Zinkevich et al. 2010) (McDonald, Hall, and Mann 2010).
Compared with parallel mini-batch SGD, the communication overhead of model
averaging is significantly reduced. Impressively, tremendous experimental works
have verified that model averaging can still achieve a good speed-up of the
training time as long as the averaging interval is carefully controlled.
However, it remains a mystery in theory why such a simple heuristic works so
well. This paper provides a thorough and rigorous theoretical study on why
model averaging can work as well as parallel mini-batch SGD with significantly
less communication overhead.
SRN: Side-output Residual Network for Object Reflection Symmetry Detection and Beyond
Authors: Wei Ke, Jie Chen, Jianbin Jiao, Guoying Zhao, Qixiang Ye
Date: 2018-07-17 00:00:00
Tasks: Edge Detection, Hand Pose Estimation, Object, Symmetry Detection

In this paper, we establish a baseline for object reflection symmetry
detection in complex backgrounds by presenting a new benchmark and an
end-to-end deep learning approach, opening up a promising direction for
symmetry detection in the wild. The new benchmark, Sym-PASCAL, spans challenges
including object diversity, multi-objects, part-invisibility, and various
complex backgrounds that are far beyond those in existing datasets. The
end-to-end deep learning approach, referred to as a side-output residual
network (SRN), leverages the output residual units (RUs) to fit the errors
between the object ground-truth symmetry and the side-outputs of multiple
stages. By cascading RUs in a deep-to-shallow manner, SRN exploits the 'flow'
of errors among multiple stages to address the challenges of fitting complex
output with limited convolutional layers, suppressing the complex backgrounds,
and effectively matching object symmetry at different scales. SRN is further
upgraded to a multi-task side-output residual network (MT-SRN) for joint
symmetry and edge detection, demonstrating its generality to image-to-mask
learning tasks. Experimental results validate both the challenging aspects of
Sym-PASCAL benchmark related to real-world images and the state-of-the-art
performance of the proposed SRN approach.
Deep Reinforcement Learning for Swarm Systems
Authors: Maximilian Hüttenrauch, Adrian Šošić, Gerhard Neumann
Date: 2018-07-17 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions. We treat the agents as samples of a distribution and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and a neural network learned end-to-end. We evaluate the representation on two well known problems from the swarm literature (rendezvous and pursuit evasion), in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. Of all approaches, the mean embedding representation using neural network features enables the richest information exchange between neighboring agents facilitating the development of more complex collective strategies.
Learning Noise-Invariant Representations for Robust Speech Recognition
Authors: Davis Liang, Zhiheng Huang, Zachary C. Lipton
Date: 2018-07-17 00:00:00
Tasks: Data Augmentation, Representation Learning, Robust Speech Recognition, speech-recognition, Speech Recognition

Despite rapid advances in speech recognition, current models remain brittle
to superficial perturbations to their inputs. Small amounts of noise can
destroy the performance of an otherwise state-of-the-art model. To harden
models against background noise, practitioners often perform data augmentation,
adding artificially-noised examples to the training set, carrying over the
original label. In this paper, we hypothesize that a clean example and its
superficially perturbed counterparts shouldn't merely map to the same class ---
they should map to the same representation. We propose
invariant-representation-learning (IRL): At each training iteration, for each
training example,we sample a noisy counterpart. We then apply a penalty term to
coerce matched representations at each layer (above some chosen layer). Our key
results, demonstrated on the Librispeech dataset are the following: (i) IRL
significantly reduces character error rates (CER) on both 'clean' (3.3% vs
6.5%) and 'other' (11.0% vs 18.1%) test sets; (ii) on several out-of-domain
noise settings (different from those seen during training), IRL's benefits are
even more pronounced. Careful ablations confirm that our results are not simply
due to shrinking activations at the chosen layers.
A Fast Segmentation-free Fully Automated Approach to White Matter Injury Detection in Preterm Infants
Authors: Subhayan Mukherjee, Irene Cheng, Steven Miller, Jessie Guo, Vann Chau, Anup Basu
Date: 2018-07-17 00:00:00
Tasks: Segmentation

White Matter Injury (WMI) is the most prevalent brain injury in the preterm
neonate leading to developmental deficits. However, detecting WMI in Magnetic
Resonance (MR) images of preterm neonate brains using traditional WM
segmentation-based methods is difficult mainly due to lack of reliable preterm
neonate brain atlases to guide segmentation. Hence, we propose a
segmentation-free, fast, unsupervised, atlas-free WMI detection method. We
detect the ventricles as blobs using a fast linear Maximally Stable Extremal
Regions algorithm. A reference contour equidistant from the blobs and the
brain-background boundary is used to identify tissue adjacent to the blobs.
Assuming normal distribution of the gray-value intensity of this tissue, the
outlier intensities in the entire brain region are identified as potential WMI
candidates. Thereafter, false positives are discriminated using appropriate
heuristics. Experiments using an expert-annotated dataset show that the
proposed method runs 20 times faster than our earlier work which relied on
time-consuming segmentation of the WM region, without compromising WMI
detection accuracy.
Data-Efficient Weakly Supervised Learning for Low-Resource Audio Event Detection Using Deep Learning
Authors: Veronica Morfi, Dan Stowell
Date: 2018-07-17 00:00:00
Tasks: Event Detection, Weakly-supervised Learning

We propose a method to perform audio event detection under the common
constraint that only limited training data are available. In training a deep
learning system to perform audio event detection, two practical problems arise.
Firstly, most datasets are "weakly labelled" having only a list of events
present in each recording without any temporal information for training.
Secondly, deep neural networks need a very large amount of labelled training
data to achieve good quality performance, yet in practice it is difficult to
collect enough samples for most classes of interest. In this paper, we propose
a data-efficient training of a stacked convolutional and recurrent neural
network. This neural network is trained in a multi instance learning setting
for which we introduce a new loss function that leads to improved training
compared to the usual approaches for weakly supervised learning. We
successfully test our approach on two low-resource datasets that lack temporal
labels.
Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)
Authors: Jungwook Choi, Pierce I-Jen Chuang, Zhuo Wang, Swagath Venkataramani, Vijayalakshmi Srinivasan, Kailash Gopalakrishnan
Date: 2018-07-17 00:00:00
Tasks: General Classification, Quantization

Deep learning algorithms achieve high classification accuracy at the expense
of significant computation cost. In order to reduce this cost, several
quantization schemes have gained attention recently with some focusing on
weight quantization, and others focusing on quantizing activations. This paper
proposes novel techniques that target weight and activation quantizations
separately resulting in an overall quantized neural network (QNN). The
activation quantization technique, PArameterized Clipping acTivation (PACT),
uses an activation clipping parameter $\alpha$ that is optimized during
training to find the right quantization scale. The weight quantization scheme,
statistics-aware weight binning (SAWB), finds the optimal scaling factor that
minimizes the quantization error based on the statistical characteristics of
the distribution of weights without the need for an exhaustive search. The
combination of PACT and SAWB results in a 2-bit QNN that achieves
state-of-the-art classification accuracy (comparable to full precision
networks) across a range of popular models and datasets.
Discrete linear-complexity reinforcement learning in continuous action spaces for Q-learning algorithms
Authors: Peyman Tavallali, Gary B. Doran Jr., Lukas Mandrake
Date: 2018-07-16 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this article, we sketch an algorithm that extends the Q-learning
algorithms to the continuous action space domain. Our method is based on the
discretization of the action space. Despite the commonly used discretization
methods, our method does not increase the discretized problem dimensionality
exponentially. We will show that our proposed method is linear in complexity
when the discretization is employed. The variant of the Q-learning algorithm
presented in this work, labeled as Finite Step Q-Learning (FSQ), can be
deployed to both shallow and deep neural network architectures.
Human Perception of Surprise: A User Study
Authors: Nalin Chhibber, Rohail Syed, Mengqiu Teng, Joslin Goh, Kevyn Collins-Thompson, Edith Law
Date: 2018-07-16 00:00:00

Understanding how to engage users is a critical question in many
applications. Previous research has shown that unexpected or astonishing events
can attract user attention, leading to positive outcomes such as engagement and
learning. In this work, we investigate the similarity and differences in how
people and algorithms rank the surprisingness of facts. Our crowdsourcing
study, involving 106 participants, shows that computational models of surprise
can be used to artificially induce surprise in humans.
Generating Synthetic Data for Neural Keyword-to-Question Models
Authors: Heng Ding, Krisztian Balog
Date: 2018-07-14 00:00:00
Tasks: Machine Translation, Translation

Search typically relies on keyword queries, but these are often semantically
ambiguous. We propose to overcome this by offering users natural language
questions, based on their keyword queries, to disambiguate their intent. This
keyword-to-question task may be addressed using neural machine translation
techniques. Neural translation models, however, require massive amounts of
training data (keyword-question pairs), which is unavailable for this task. The
main idea of this paper is to generate large amounts of synthetic training data
from a small seed set of hand-labeled keyword-question pairs. Since natural
language questions are available in large quantities, we develop models to
automatically generate the corresponding keyword queries. Further, we introduce
various filtering mechanisms to ensure that synthetic training data is of high
quality. We demonstrate the feasibility of our approach using both automatic
and manual evaluation. This is an extended version of the article published
with the same title in the Proceedings of ICTIR'18.
Machine Learning Classifiers Do Not Improve the Prediction of Academic Risk: Evidence from Australia
Authors: Sarah Cornell-Farrow, Robert Garrard
Date: 2018-07-19 00:00:00
Tasks: BIG-bench Machine Learning, regression

Machine learning methods tend to outperform traditional statistical models at prediction. In the prediction of academic achievement, ML models have not shown substantial improvement over logistic regression. So far, these results have almost entirely focused on college achievement, due to the availability of administrative datasets, and have contained relatively small sample sizes by ML standards. In this article we apply popular machine learning models to a large dataset ($n=1.2$ million) containing primary and middle school performance on a standardized test given annually to Australian students. We show that machine learning models do not outperform logistic regression for detecting students who will perform in the `below standard' band of achievement upon sitting their next test, even in a large-$n$ setting.
Emulating malware authors for proactive protection using GANs over a distributed image visualization of dynamic file behavior
Authors: Vineeth S. Bhaskara, Debanjan Bhattacharyya
Date: 2018-07-19 00:00:00
Tasks: Generative Adversarial Network

Malware authors have always been at an advantage of being able to
adversarially test and augment their malicious code, before deploying the
payload, using anti-malware products at their disposal. The anti-malware
developers and threat experts, on the other hand, do not have such a privilege
of tuning anti-malware products against zero-day attacks pro-actively. This
allows the malware authors to being a step ahead of the anti-malware products,
fundamentally biasing the cat and mouse game played by the two parties. In this
paper, we propose a way that would enable machine learning based threat
prevention models to bridge that gap by being able to tune against a deep
generative adversarial network (GAN), which takes up the role of a malware
author and generates new types of malware. The GAN is trained over a reversible
distributed RGB image representation of known malware behaviors, encoding the
sequence of API call ngrams and the corresponding term frequencies. The
generated images represent synthetic malware that can be decoded back to the
underlying API call sequence information. The image representation is not only
demonstrated as a general technique of incorporating necessary priors for
exploiting convolutional neural network architectures for generative or
discriminative modeling, but also as a visualization method for easy manual
software or malware categorization, by having individual API ngram information
distributed across the image space. In addition, we also propose using
smart-definitions for detecting malwares based on perceptual hashing of these
images. Such hashes are potentially more effective than cryptographic hashes
that do not carry any meaningful similarity metric, and hence, do not
generalize well.