pwc-archive/papers-with-abstracts huggingface dataset:
Improving the accuracy of nearest-neighbor classification using principled construction and stochastic sampling of training-set centroids
Authors: Stephen Whitelam
Date: 2018-09-07 00:00:00
Tasks: Classification, General Classification

A conceptually simple way to classify images is to directly compare test-set data and training-set data. The accuracy of this approach is limited by the method of comparison used, and by the extent to which the training-set data cover configuration space. Here we show that this coverage can be substantially increased using coarse graining (replacing groups of images by their centroids) and stochastic sampling (using distinct sets of centroids in combination). We use the MNIST and Fashion-MNIST data sets to show that a principled coarse-graining algorithm can convert training images into fewer image centroids without loss of accuracy of classification of test-set images by nearest-neighbor classification. Distinct batches of centroids can be used in combination as a means of stochastically sampling configuration space, and can classify test-set data more accurately than can the unaltered training set. On the MNIST and Fashion-MNIST data sets this approach converts nearest-neighbor classification from a mid-ranking- to an upper-ranking member of the set of classical machine-learning techniques.
VOS: a Method for Variational Oversampling of Imbalanced Data
Authors: Val Andrei Fajardo, David Findlay, Roshanak Houmanfar, Charu Jaiswal, Jiaxi Liang, Honglei Xie
Date: 2018-09-07 00:00:00
Tasks: Fraud Detection, General Classification, Variational Inference

Class imbalanced datasets are common in real-world applications that range
from credit card fraud detection to rare disease diagnostics. Several popular
classification algorithms assume that classes are approximately balanced, and
hence build the accompanying objective function to maximize an overall accuracy
rate. In these situations, optimizing the overall accuracy will lead to highly
skewed predictions towards the majority class. Moreover, the negative business
impact resulting from false positives (positive samples incorrectly classified
as negative) can be detrimental. Many methods have been proposed to address the
class imbalance problem, including methods such as over-sampling,
under-sampling and cost-sensitive methods. In this paper, we consider the
over-sampling method, where the aim is to augment the original dataset with
synthetically created observations of the minority classes. In particular,
inspired by the recent advances in generative modelling techniques (e.g.,
Variational Inference and Generative Adversarial Networks), we introduce a new
oversampling technique based on variational autoencoders. Our experiments show
that the new method is superior in augmenting datasets for downstream
classification tasks when compared to traditional oversampling methods.
Logographic Subword Model for Neural Machine Translation
Authors: Yihao Fang, Rong Zheng, Xiaodan Zhu
Date: 2018-09-07 00:00:00
Tasks: Machine Translation, model, Translation

A novel logographic subword model is proposed to reinterpret logograms as
abstract subwords for neural machine translation. Our approach drastically
reduces the size of an artificial neural network, while maintaining comparable
BLEU scores as those attained with the baseline RNN and CNN seq2seq models. The
smaller model size also leads to shorter training and inference time.
Experiments demonstrate that in the tasks of English-Chinese/Chinese-English
translation, the reduction of those aspects can be from $11\%$ to as high as
$77\%$. Compared to previous subword models, abstract subwords can be applied
to various logographic languages. Considering most of the logographic languages
are ancient and very low resource languages, these advantages are very
desirable for archaeological computational linguistic applications such as a
resource-limited offline hand-held Demotic-English translator.
Learning Invariances for Policy Generalization
Authors: Remi Tachet, Philip Bachman, Harm van Seijen
Date: 2018-09-07 00:00:00
Tasks: BIG-bench Machine Learning, Data Augmentation, Meta-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

While recent progress has spawned very powerful machine learning systems, those agents remain extremely specialized and fail to transfer the knowledge they gain to similar yet unseen tasks. In this paper, we study a simple reinforcement learning problem and focus on learning policies that encode the proper invariances for generalization to different settings. We evaluate three potential methods for policy generalization: data augmentation, meta-learning and adversarial training. We find our data augmentation method to be effective, and study the potential of meta-learning and adversarial learning as alternative task-agnostic approaches.
HyperGCN: A New Method of Training Graph Convolutional Networks on Hypergraphs
Authors: Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, Partha Talukdar
Date: 2018-09-07 00:00:00

In many real-world network datasets such as co-authorship, co-citation, email communication, etc., relationships are complex and go beyond pairwise. Hypergraphs provide a flexible and natural modeling tool to model such complex relationships. The obvious existence of such complex relationships in many real-world networks naturaly motivates the problem of learning with hypergraphs. A popular learning paradigm is hypergraph-based semi-supervised learning (SSL) where the goal is to assign labels to initially unlabeled vertices in a hypergraph. Motivated by the fact that a graph convolutional network (GCN) has been effective for graph-based SSL, we propose HyperGCN, a novel GCN for SSL on attributed hypergraphs. Additionally, we show how HyperGCN can be used as a learning-based approach for combinatorial optimisation on NP-hard hypergraph problems. We demonstrate HyperGCN's effectiveness through detailed experimentation on real-world hypergraphs.
Self-Supervised Generation of Spatial Audio for 360 Video
Authors: Pedro Morgado, Nuno Vasconcelos, Timothy Langlois, Oliver Wang
Date: 2018-09-07 00:00:00

We introduce an approach to convert mono audio recorded by a 360 video camera
into spatial audio, a representation of the distribution of sound over the full
viewing sphere. Spatial audio is an important component of immersive 360 video
viewing, but spatial audio microphones are still rare in current 360 video
production. Our system consists of end-to-end trainable neural networks that
separate individual sound sources and localize them on the viewing sphere,
conditioned on multi-modal analysis of audio and 360 video frames. We introduce
several datasets, including one filmed ourselves, and one collected in-the-wild
from YouTube, consisting of 360 videos uploaded with spatial audio. During
training, ground-truth spatial audio serves as self-supervision and a mixed
down mono track forms the input to our network. Using our approach, we show
that it is possible to infer the spatial location of sound sources based only
on 360 video and a mono audio track.
The largest cognitive systems will be optoelectronic
Authors: Jeffrey M. Shainline
Date: 2018-09-07 00:00:00

Electrons and photons offer complementary strengths for information
processing. Photons are excellent for communication, while electrons are
superior for computation and memory. Cognition requires distributed computation
to be communicated across the system for information integration. We present
reasoning from neuroscience, network theory, and device physics supporting the
conjecture that large-scale cognitive systems will benefit from electronic
devices performing synaptic, dendritic, and neuronal information processing
operating in conjunction with photonic communication. On the chip scale,
integrated dielectric waveguides enable fan-out to thousands of connections. On
the system scale, fiber and free-space optics can be employed. The largest
cognitive systems will be limited by the distance light can travel during the
period of a network oscillation. We calculate that optoelectronic networks the
area of a large data center ($10^5$\,m$^2$) will be capable of system-wide
information integration at $1$\,MHz. At frequencies of cortex-wide integration
in the human brain ($4$\,Hz, theta band), optoelectronic systems could
integrate information across the surface of the earth.
Skin lesion classification with ensemble of squeeze-and-excitation networks and semi-supervised learning
Authors: Shunsuke Kitada, Hitoshi Iyatomi
Date: 2018-09-07 00:00:00
Tasks: Classification, Data Augmentation, General Classification, Lesion Classification, Skin Lesion Classification

In this report, we introduce the outline of our system in Task 3: Disease
Classification of ISIC 2018: Skin Lesion Analysis Towards Melanoma Detection.
We fine-tuned multiple pre-trained neural network models based on
Squeeze-and-Excitation Networks (SENet) which achieved state-of-the-art results
in the field of image recognition. In addition, we used the mean teachers as a
semi-supervised learning framework and introduced some specially designed data
augmentation strategies for skin lesion analysis. We confirmed our data
augmentation strategy improved classification performance and demonstrated
87.2% in balanced accuracy on the official ISIC2018 validation dataset.
A Deeper Look at 3D Shape Classifiers
Authors: Jong-Chyi Su, Matheus Gadelha, Rui Wang, Subhransu Maji
Date: 2018-09-07 00:00:00
Tasks: 3D Shape Classification, Computational Efficiency, Transfer Learning

We investigate the role of representations and architectures for classifying
3D shapes in terms of their computational efficiency, generalization, and
robustness to adversarial transformations. By varying the number of training
examples and employing cross-modal transfer learning we study the role of
initialization of existing deep architectures for 3D shape classification. Our
analysis shows that multiview methods continue to offer the best generalization
even without pretraining on large labeled image datasets, and even when trained
on simplified inputs such as binary silhouettes. Furthermore, the performance
of voxel-based 3D convolutional networks and point-based architectures can be
improved via cross-modal transfer from image representations. Finally, we
analyze the robustness of 3D shape classifiers to adversarial transformations
and present a novel approach for generating adversarial perturbations of a 3D
shape for multiview classifiers using a differentiable renderer. We find that
point-based networks are more robust to point position perturbations while
voxel-based and multiview networks are easily fooled with the addition of
imperceptible noise to the input.
Using Sparse Semantic Embeddings Learned from Multimodal Text and Image Data to Model Human Conceptual Knowledge
Authors: Steven Derby, Paul Miller, Brian Murphy, Barry Devereux
Date: 2018-09-07 00:00:00
Proceeding: CONLL 2018 10

Distributional models provide a convenient way to model semantics using dense
embedding spaces derived from unsupervised learning algorithms. However, the
dimensions of dense embedding spaces are not designed to resemble human
semantic knowledge. Moreover, embeddings are often built from a single source
of information (typically text data), even though neurocognitive research
suggests that semantics is deeply linked to both language and perception. In
this paper, we combine multimodal information from both text and image-based
representations derived from state-of-the-art distributional models to produce
sparse, interpretable vectors using Joint Non-Negative Sparse Embedding.
Through in-depth analyses comparing these sparse models to human-derived
behavioural and neuroimaging data, we demonstrate their ability to predict
interpretable linguistic descriptions of human ground-truth semantic knowledge.
Fairness Through Causal Awareness: Learning Latent-Variable Models for Biased Data
Authors: David Madras, Elliot Creager, Toniann Pitassi, Richard Zemel
Date: 2018-09-07 00:00:00
Tasks: Attribute, Fairness, General Classification

How do we learn from biased data? Historical datasets often reflect
historical prejudices; sensitive or protected attributes may affect the
observed treatments and outcomes. Classification algorithms tasked with
predicting outcomes accurately from these datasets tend to replicate these
biases. We advocate a causal modeling approach to learning from biased data,
exploring the relationship between fair classification and intervention. We
propose a causal model in which the sensitive attribute confounds both the
treatment and the outcome. Building on prior work in deep learning and
generative modeling, we describe how to learn the parameters of this causal
model from observational data alone, even in the presence of unobserved
confounders. We show experimentally that fairness-aware causal modeling
provides better estimates of the causal effects between the sensitive
attribute, the treatment, and the outcome. We further present evidence that
estimating these causal effects can help learn policies that are both more
accurate and fair, when presented with a historically biased dataset.
Multi-level hypothesis testing for populations of heterogeneous networks
Authors: Guilherme Gomes, Vinayak Rao, Jennifer Neville
Date: 2018-09-07 00:00:00
Tasks: Anomaly Detection, Two-sample testing

In this work, we consider hypothesis testing and anomaly detection on
datasets where each observation is a weighted network. Examples of such data
include brain connectivity networks from fMRI flow data, or word co-occurrence
counts for populations of individuals. Current approaches to hypothesis testing
for weighted networks typically requires thresholding the edge-weights, to
transform the data to binary networks. This results in a loss of information,
and outcomes are sensitivity to choice of threshold levels. Our work avoids
this, and we consider weighted-graph observations in two situations, 1) where
each graph belongs to one of two populations, and 2) where entities belong to
one of two populations, with each entity possessing multiple graphs (indexed
e.g. by time). Specifically, we propose a hierarchical Bayesian hypothesis
testing framework that models each population with a mixture of latent space
models for weighted networks, and then tests populations of networks for
differences in distribution over components. Our framework is capable of
population-level, entity-specific, as well as edge-specific hypothesis testing.
We apply it to synthetic data and three real-world datasets: two social media
datasets involving word co-occurrences from discussions on Twitter of the
political unrest in Brazil, and on Instagram concerning Attention Deficit
Hyperactivity Disorder (ADHD) and depression drugs, and one medical dataset
involving fMRI brain-scans of human subjects. The results show that our
proposed method has lower Type I error and higher statistical power compared to
alternatives that need to threshold the edge weights. Moreover, they show our
proposed method is better suited to deal with highly heterogeneous datasets.
MixUp as Locally Linear Out-Of-Manifold Regularization
Authors: Hongyu Guo, Yongyi Mao, Richong Zhang
Date: 2018-09-07 00:00:00
Tasks: Data Augmentation

MixUp is a recently proposed data-augmentation scheme, which linearly
interpolates a random pair of training examples and correspondingly the one-hot
representations of their labels. Training deep neural networks with such
additional data is shown capable of significantly improving the predictive
accuracy of the current art. The power of MixUp, however, is primarily
established empirically and its working and effectiveness have not been
explained in any depth. In this paper, we develop an understanding for MixUp as
a form of "out-of-manifold regularization", which imposes certain "local
linearity" constraints on the model's input space beyond the data manifold.
This analysis enables us to identify a limitation of MixUp, which we call
"manifold intrusion". In a nutshell, manifold intrusion in MixUp is a form of
under-fitting resulting from conflicts between the synthetic labels of the
mixed-up examples and the labels of original training data. Such a phenomenon
usually happens when the parameters controlling the generation of mixing
policies are not sufficiently fine-tuned on the training data. To address this
issue, we propose a novel adaptive version of MixUp, where the mixing policies
are automatically learned from the data using an additional network and
objective function designed to avoid manifold intrusion. The proposed
regularizer, AdaMixUp, is empirically evaluated on several benchmark datasets.
Extensive experiments demonstrate that AdaMixUp improves upon MixUp when
applied to the current art of deep classification models.
Sparse Kernel PCA for Outlier Detection
Authors: Rudrajit Das, Aditya Golatkar, Suyash P. Awate
Date: 2018-09-07 00:00:00
Tasks: Outlier Detection

In this paper, we propose a new method to perform Sparse Kernel Principal
Component Analysis (SKPCA) and also mathematically analyze the validity of
SKPCA. We formulate SKPCA as a constrained optimization problem with elastic
net regularization (Hastie et al.) in kernel feature space and solve it. We
consider outlier detection (where KPCA is employed) as an application for
SKPCA, using the RBF kernel. We test it on 5 real-world datasets and show that
by using just 4% (or even less) of the principal components (PCs), where each
PC has on average less than 12% non-zero elements in the worst case among all 5
datasets, we are able to nearly match and in 3 datasets even outperform KPCA.
We also compare the performance of our method with a recently proposed method
for SKPCA by Wang et al. and show that our method performs better in terms of
both accuracy and sparsity. We also provide a novel probabilistic proof to
justify the existence of sparse solutions for KPCA using the RBF kernel. To the
best of our knowledge, this is the first attempt at theoretically analyzing the
validity of SKPCA.
Meteorologists and Students: A resource for language grounding of geographical descriptors
Authors: Alejandro Ramos-Soto, Ehud Reiter, Kees Van Deemter, Jose M. Alonso, Albert Gatt
Date: 2018-09-07 00:00:00
Proceeding: WS 2018 11
Tasks: Referring Expression, Referring expression generation

We present a data resource which can be useful for research purposes on
language grounding tasks in the context of geographical referring expression
generation. The resource is composed of two data sets that encompass 25
different geographical descriptors and a set of associated graphical
representations, drawn as polygons on a map by two groups of human subjects:
teenage students and expert meteorologists.
BiasedWalk: Biased Sampling for Representation Learning on Graphs
Authors: Duong Nguyen, Fragkiskos D. Malliaros
Date: 2018-09-07 00:00:00
Tasks: Community Detection, General Classification, Link Prediction, Network Embedding, Node Classification, Representation Learning

Network embedding algorithms are able to learn latent feature representations
of nodes, transforming networks into lower dimensional vector representations.
Typical key applications, which have effectively been addressed using network
embeddings, include link prediction, multilabel classification and community
detection. In this paper, we propose BiasedWalk, a scalable, unsupervised
feature learning algorithm that is based on biased random walks to sample
context information about each node in the network. Our random-walk based
sampling can behave as Breath-First-Search (BFS) and Depth-First-Search (DFS)
samplings with the goal to capture homophily and role equivalence between the
nodes in the network. We have performed a detailed experimental evaluation
comparing the performance of the proposed algorithm against various baseline
methods, on several datasets and learning tasks. The experiment results show
that the proposed method outperforms the baseline ones in most of the tasks and
datasets.
Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer
Authors: Alvin Chan, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Yang Liu, Yew Soon Ong
Date: 2018-09-07 00:00:00
Tasks: Question Answering, Relation

Deep neural networks (DNN), while becoming the driving force of many novel
technology and achieving tremendous success in many cutting-edge applications,
are still vulnerable to adversarial attacks. Differentiable neural computer
(DNC) is a novel computing machine with DNN as its central controller operating
on an external memory module for data processing. The unique architecture of
DNC contributes to its state-of-the-art performance in tasks which requires the
ability to represent variables and data structure as well as to store data over
long timescales. However, there still lacks a comprehensive study on how
adversarial examples affect DNC in terms of robustness. In this paper, we
propose metamorphic relation based adversarial techniques for a range of tasks
described in the natural processing language domain. We show that the
near-perfect performance of the DNC in bAbI logical question answering tasks
can be degraded by adversarially injected sentences. We further perform
in-depth study on the role of DNC's memory size in its robustness and analyze
the potential reason causing why DNC fails. Our study demonstrates the current
challenges and potential opportunities towards constructing more robust DNCs.
StackNet: Stacking Parameters for Continual learning
Authors: Jangho Kim, Jeesoo Kim, Nojun Kwak
Date: 2018-09-07 00:00:00
Tasks: Continual Learning

Training a neural network for a classification task typically assumes that the data to train are given from the beginning. However, in the real world, additional data accumulate gradually and the model requires additional training without accessing the old training data. This usually leads to the catastrophic forgetting problem which is inevitable for the traditional training methodology of neural networks. In this paper, we propose a continual learning method that is able to learn additional tasks while retaining the performance of previously learned tasks by stacking parameters. Composed of two complementary components, the index module and the StackNet, our method estimates the index of the corresponding task for an input sample with the index module and utilizes a particular portion of StackNet with this index. The StackNet guarantees no degradation in the performance of the previously learned tasks and the index module shows high confidence in finding the origin of an input sample. Compared to the previous work of PackNet, our method is competitive and highly intuitive.
Optimizing deep video representation to match brain activity
Authors: Hugo Richard, Ana Pinho, Bertrand Thirion, Guillaume Charpiat
Date: 2018-09-07 00:00:00
Tasks: Optical Flow Estimation

The comparison of observed brain activity with the statistics generated by
artificial intelligence systems is useful to probe brain functional
organization under ecological conditions. Here we study fMRI activity in ten
subjects watching color natural movies and compute deep representations of
these movies with an architecture that relies on optical flow and image
content. The association of activity in visual areas with the different layers
of the deep architecture displays complexity-related contrasts across visual
areas and reveals a striking foveal/peripheral dichotomy.