pwc-archive/papers-with-abstracts huggingface dataset:
A Grounded Unsupervised Universal Part-of-Speech Tagger for Low-Resource Languages
Authors: Ronald Cardenas, Ying Lin, Heng Ji, Jonathan May
Date: 2019-04-10 00:00:00
Proceeding: NAACL 2019 6
Tasks: Clustering, Decipherment, Part-Of-Speech Tagging, POS, POS Tagging, TAG

Unsupervised part of speech (POS) tagging is often framed as a clustering
problem, but practical taggers need to \textit{ground} their clusters as well.
Grounding generally requires reference labeled data, a luxury a low-resource
language might not have. In this work, we describe an approach for low-resource
unsupervised POS tagging that yields fully grounded output and requires no
labeled training data. We find the classic method of Brown et al. (1992)
clusters well in our use case and employ a decipherment-based approach to
grounding. This approach presumes a sequence of cluster IDs is a `ciphertext'
and seeks a POS tag-to-cluster ID mapping that will reveal the POS sequence. We
show intrinsically that, despite the difficulty of the task, we obtain
reasonable performance across a variety of languages. We also show
extrinsically that incorporating our POS tagger into a name tagger leads to
state-of-the-art tagging performance in Sinhalese and Kinyarwanda, two
languages with nearly no labeled POS data available. We further demonstrate our
tagger's utility by incorporating it into a true `zero-resource' variant of the
Malopa (Ammar et al., 2016) dependency parser model that removes the current
reliance on multilingual resources and gold POS tags for new languages.
Experiments show that including our tagger makes up much of the accuracy lost
when gold POS tags are unavailable.
The Weight Function in the Subtree Kernel is Decisive
Authors: Romain Azaïs, Florian Ingels
Date: 2019-04-10 00:00:00

Tree data are ubiquitous because they model a large variety of situations, e.g., the architecture of plants, the secondary structure of RNA, or the hierarchy of XML files. Nevertheless, the analysis of these non-Euclidean data is difficult per se. In this paper, we focus on the subtree kernel that is a convolution kernel for tree data introduced by Vishwanathan and Smola in the early 2000's. More precisely, we investigate the influence of the weight function from a theoretical perspective and in real data applications. We establish on a 2-classes stochastic model that the performance of the subtree kernel is improved when the weight of leaves vanishes, which motivates the definition of a new weight function, learned from the data and not fixed by the user as usually done. To this end, we define a unified framework for computing the subtree kernel from ordered or unordered trees, that is particularly suitable for tuning parameters. We show through eight real data classification problems the great efficiency of our approach, in particular for small datasets, which also states the high importance of the weight function. Finally, a visualization tool of the significant features is derived.
FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning
Authors: Ángel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, Duen Horng Chau
Date: 2019-04-10 00:00:00
Tasks: BIG-bench Machine Learning, Fairness, Subgroup Discovery

The growing capability and accessibility of machine learning has led to its application to many real-world domains and data about people. Despite the benefits algorithmic systems may bring, models can reflect, inject, or exacerbate implicit and explicit societal biases into their outputs, disadvantaging certain demographic subgroups. Discovering which biases a machine learning model has introduced is a great challenge, due to the numerous definitions of fairness and the large number of potentially impacted subgroups. We present FairVis, a mixed-initiative visual analytics system that integrates a novel subgroup discovery technique for users to audit the fairness of machine learning models. Through FairVis, users can apply domain knowledge to generate and investigate known subgroups, and explore suggested and similar subgroups. FairVis' coordinated views enable users to explore a high-level overview of subgroup performance and subsequently drill down into detailed investigation of specific subgroups. We show how FairVis helps to discover biases in two real datasets used in predicting income and recidivism. As a visual analytics system devoted to discovering bias in machine learning, FairVis demonstrates how interactive visualization may help data scientists and the general public understand and create more equitable algorithmic systems.
Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems
Authors: Leah Bar, Nir Sochen
Date: 2019-04-10 00:00:00

We propose a neural network-based algorithm for solving forward and inverse
problems for partial differential equations in unsupervised fashion. The
solution is approximated by a deep neural network which is the minimizer of a
cost function, and satisfies the PDE, boundary conditions, and additional
regularizations. The method is mesh free and can be easily applied to an
arbitrary regular domain. We focus on 2D second order elliptical system with
non-constant coefficients, with application to Electrical Impedance Tomography.
Deep Learning for System Trace Restoration
Authors: Ilia Sucholutsky, Apurva Narayan, Matthias Schonlau, Sebastian Fischmeister
Date: 2019-04-10 00:00:00
Tasks: Anomaly Detection, Deep Learning

Most real-world datasets, and particularly those collected from physical
systems, are full of noise, packet loss, and other imperfections. However, most
specification mining, anomaly detection and other such algorithms assume, or
even require, perfect data quality to function properly. Such algorithms may
work in lab conditions when given clean, controlled data, but will fail in the
field when given imperfect data. We propose a method for accurately
reconstructing discrete temporal or sequential system traces affected by data
loss, using Long Short-Term Memory Networks (LSTMs). The model works by
learning to predict the next event in a sequence of events, and uses its own
output as an input to continue predicting future events. As a result, this
method can be used for data restoration even with streamed data. Such a method
can reconstruct even long sequence of missing events, and can also help
validate and improve data quality for noisy data. The output of the model will
be a close reconstruction of the true data, and can be fed to algorithms that
rely on clean data. We demonstrate our method by reconstructing automotive CAN
traces consisting of long sequences of discrete events. We show that given even
small parts of a CAN trace, our LSTM model can predict future events with an
accuracy of almost 90%, and can successfully reconstruct large portions of the
original trace, greatly outperforming a Markov Model benchmark. We separately
feed the original, lossy, and reconstructed traces into a specification mining
framework to perform downstream analysis of the effect of our method on
state-of-the-art models that use these traces for understanding the behavior of
complex systems.
Attentive Action and Context Factorization
Authors: Yang Wang, Vinh Tran, Gedas Bertasius, Lorenzo Torresani, Minh Hoai
Date: 2019-04-10 00:00:00
Tasks: Action Recognition, Temporal Action Localization

We propose a method for human action recognition, one that can localize the
spatiotemporal regions that `define' the actions. This is a challenging task
due to the subtlety of human actions in video and the co-occurrence of
contextual elements. To address this challenge, we utilize conjugate samples of
human actions, which are video clips that are contextually similar to human
action samples but do not contain the action. We introduce a novel attentional
mechanism that can spatially and temporally separate human actions from the
co-occurring contextual factors. The separation of the action and context
factors is weakly supervised, eliminating the need for laboriously detailed
annotation of these two factors in training samples. Our method can be used to
build human action classifiers with higher accuracy and better
interpretability. Experiments on several human action recognition datasets
demonstrate the quantitative and qualitative benefits of our approach.
Sliced Wasserstein Generative Models
Authors: Jiqing Wu, Zhiwu Huang, Dinesh Acharya, Wen Li, Janine Thoma, Danda Pani Paudel, Luc van Gool
Date: 2019-04-10 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Generation, Video Generation

In generative modeling, the Wasserstein distance (WD) has emerged as a useful
metric to measure the discrepancy between generated and real data
distributions. Unfortunately, it is challenging to approximate the WD of
high-dimensional distributions. In contrast, the sliced Wasserstein distance
(SWD) factorizes high-dimensional distributions into their multiple
one-dimensional marginal distributions and is thus easier to approximate. In
this paper, we introduce novel approximations of the primal and dual SWD.
Instead of using a large number of random projections, as it is done by
conventional SWD approximation methods, we propose to approximate SWDs with a
small number of parameterized orthogonal projections in an end-to-end deep
learning fashion. As concrete applications of our SWD approximations, we design
two types of differentiable SWD blocks to equip modern generative
frameworks---Auto-Encoders (AE) and Generative Adversarial Networks (GAN). In
the experiments, we not only show the superiority of the proposed generative
models on standard image synthesis benchmarks, but also demonstrate the
state-of-the-art performance on challenging high resolution image and video
generation in an unsupervised manner.
Constructing Clustering Transformations
Authors: Steffen Borgwardt, Charles Viss
Date: 2019-04-10 00:00:00
Tasks: Clustering

Clustering is one of the fundamental tasks in data analytics and machine learning. In many situations, different clusterings of the same data set become relevant. For example, different algorithms for the same clustering task may return dramatically different solutions. We are interested in applications in which one clustering has to be transformed into another; e.g., when a gradual transition from an old solution to a new one is required. In this paper, we devise methods for constructing such a transition based on linear programming and network theory. We use a so-called clustering-difference graph to model the desired transformation and provide methods for decomposing the graph into a sequence of elementary moves that accomplishes the transformation. These moves are equivalent to the edge directions, or circuits, of the underlying partition polytopes. Therefore, in addition to a conceptually new metric for measuring the distance between clusterings, we provide new bounds on the circuit diameter of these partition polytopes.
MODL: A Modular Ontology Design Library
Authors: Cogan Shimizu, Quinn Hirt, Pascal Hitzler
Date: 2019-04-10 00:00:00

Pattern-based, modular ontologies have several beneficial properties that
lend themselves to FAIR data practices, especially as it pertains to
Interoperability and Reusability. However, developing such ontologies has a
high upfront cost, e.g. reusing a pattern is predicated upon being aware of its
existence in the first place. Thus, to help overcome these barriers, we have
developed MODL: a modular ontology design library. MODL is a curated collection
of well-documented ontology design patterns, drawn from a wide variety of
interdisciplinary use-cases. In this paper we present MODL as a resource,
discuss its use, and provide some examples of its contents.
Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spheres
Authors: Shuai Liao, Efstratios Gavves, Cees G. M. Snoek
Date: 2019-04-10 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Rotation Estimation, regression, Surface Normal Estimation, Surface Normals Estimation, Viewpoint Estimation

Many computer vision challenges require continuous outputs, but tend to be
solved by discrete classification. The reason is classification's natural
containment within a probability $n$-simplex, as defined by the popular softmax
activation function. Regular regression lacks such a closed geometry, leading
to unstable training and convergence to suboptimal local minima. Starting from
this insight we revisit regression in convolutional neural networks. We observe
many continuous output problems in computer vision are naturally contained in
closed geometrical manifolds, like the Euler angles in viewpoint estimation or
the normals in surface normal estimation. A natural framework for posing such
continuous output problems are $n$-spheres, which are naturally closed
geometric manifolds defined in the $\mathbb{R}^{(n+1)}$ space. By introducing a
spherical exponential mapping on $n$-spheres at the regression output, we
obtain well-behaved gradients, leading to stable training. We show how our
spherical regression can be utilized for several computer vision challenges,
specifically viewpoint estimation, surface normal estimation and 3D rotation
estimation. For all these problems our experiments demonstrate the benefit of
spherical regression. All paper resources are available at
https://github.com/leoshine/Spherical_Regression.
Enhancing Decision Tree based Interpretation of Deep Neural Networks through L1-Orthogonal Regularization
Authors: Nina Schaaf, Marco F. Huber, Johannes Maucher
Date: 2019-04-10 00:00:00

One obstacle that so far prevents the introduction of machine learning models primarily in critical areas is the lack of explainability. In this work, a practicable approach of gaining explainability of deep artificial neural networks (NN) using an interpretable surrogate model based on decision trees is presented. Simply fitting a decision tree to a trained NN usually leads to unsatisfactory results in terms of accuracy and fidelity. Using L1-orthogonal regularization during training, however, preserves the accuracy of the NN, while it can be closely approximated by small decision trees. Tests with different data sets confirm that L1-orthogonal regularization yields models of lower complexity and at the same time higher fidelity compared to other regularizers.
Deep Learning without Weight Transport
Authors: Mohamed Akrout, Collin Wilson, Peter C. Humphreys, Timothy Lillicrap, Douglas Tweed
Date: 2019-04-10 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Deep Learning

Current algorithms for deep learning probably cannot run in the brain because they rely on weight transport, where forward-path neurons transmit their synaptic weights to a feedback path, in a way that is likely impossible biologically. An algorithm called feedback alignment achieves deep learning without weight transport by using random feedback weights, but it performs poorly on hard visual-recognition tasks. Here we describe two mechanisms - a neural circuit called a weight mirror and a modification of an algorithm proposed by Kolen and Pollack in 1994 - both of which let the feedback path learn appropriate synaptic weights quickly and accurately even in large networks, without weight transport or complex wiring.Tested on the ImageNet visual-recognition task, these mechanisms outperform both feedback alignment and the newer sign-symmetry method, and nearly match backprop, the standard algorithm of deep learning, which uses weight transport.
Feature Engineering for Mid-Price Prediction with Deep Learning
Authors: Adamantios Ntakaris, Giorgio Mirone, Juho Kanniainen, Moncef Gabbouj, Alexandros Iosifidis
Date: 2019-04-10 00:00:00
Tasks: Deep Learning, Feature Engineering

Mid-price movement prediction based on limit order book (LOB) data is a challenging task due to the complexity and dynamics of the LOB. So far, there have been very limited attempts for extracting relevant features based on LOB data. In this paper, we address this problem by designing a new set of handcrafted features and performing an extensive experimental evaluation on both liquid and illiquid stocks. More specifically, we implement a new set of econometrical features that capture statistical properties of the underlying securities for the task of mid-price prediction. Moreover, we develop a new experimental protocol for online learning that treats the task as a multi-objective optimization problem and predicts i) the direction of the next price movement and ii) the number of order book events that occur until the change takes place. In order to predict the mid-price movement, the features are fed into nine different deep learning models based on multi-layer perceptrons (MLP), convolutional neural networks (CNN) and long short-term memory (LSTM) neural networks. The performance of the proposed method is then evaluated on liquid and illiquid stocks, which are based on TotalView-ITCH US and Nordic stocks, respectively. For some stocks, results suggest that the correct choice of a feature set and a model can lead to the successful prediction of how long it takes to have a stock price movement.
ReinBo: Machine Learning pipeline search and configuration with Bayesian Optimization embedded Reinforcement Learning
Authors: Xudong Sun, Jiali Lin, Bernd Bischl
Date: 2019-04-10 00:00:00
Tasks: Bayesian Optimization, BIG-bench Machine Learning, Feature Engineering, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Machine learning pipeline potentially consists of several stages of
operations like data preprocessing, feature engineering and machine learning
model training. Each operation has a set of hyper-parameters, which can become
irrelevant for the pipeline when the operation is not selected. This gives rise
to a hierarchical conditional hyper-parameter space. To optimize this mixed
continuous and discrete conditional hierarchical hyper-parameter space, we
propose an efficient pipeline search and configuration algorithm which combines
the power of Reinforcement Learning and Bayesian Optimization. Empirical
results show that our method performs favorably compared to state of the art
methods like Auto-sklearn , TPOT, Tree Parzen Window, and Random Search.
Scanner Invariant Representations for Diffusion MRI Harmonization
Authors: Daniel Moyer, Greg Ver Steeg, Chantal M. W. Tax, Paul M. Thompson
Date: 2019-04-10 00:00:00
Tasks: Diffusion  MRI, Fairness, Image Reconstruction

Purpose: In the present work we describe the correction of diffusion-weighted MRI for site and scanner biases using a novel method based on invariant representation. Theory and Methods: Pooled imaging data from multiple sources are subject to variation between the sources. Correcting for these biases has become very important as imaging studies increase in size and multi-site cases become more common. We propose learning an intermediate representation invariant to site/protocol variables, a technique adapted from information theory-based algorithmic fairness; by leveraging the data processing inequality, such a representation can then be used to create an image reconstruction that is uninformative of its original source, yet still faithful to underlying structures. To implement this, we use a deep learning method based on variational auto-encoders (VAE) to construct scanner invariant encodings of the imaging data. Results: To evaluate our method, we use training data from the 2018 MICCAI Computational Diffusion MRI (CDMRI) Challenge Harmonization dataset. Our proposed method shows improvements on independent test data relative to a recently published baseline method on each subtask, mapping data from three different scanning contexts to and from one separate target scanning context. Conclusion: As imaging studies continue to grow, the use of pooled multi-site imaging will similarly increase. Invariant representation presents a strong candidate for the harmonization of these data.
Searching Heterogeneous Personal Digital Traces
Authors: Daniela Vianna, Varvara Kalokyri, Alexander Borgida, Thu D. Nguyen, Amelie Marian
Date: 2019-04-10 00:00:00
Tasks: Natural Questions

Digital traces of our lives are now constantly produced by various connected
devices, internet services and interactions. Our actions result in a multitude
of heterogeneous data objects, or traces, kept in various locations in the
cloud or on local devices. Users have very few tools to organize, understand,
and search the digital traces they produce. We propose a simple but flexible
data model to aggregate, organize, and find personal information within a
collection of a user's personal digital traces. Our model uses as basic
dimensions the six questions: what, when, where, who, why, and how. These
natural questions model universal aspects of a personal data collection and
serve as unifying features of each personal data object, regardless of its
source. We propose indexing and search techniques to aid users in searching for
their past information in their unified personal digital data sets using our
model. Experiments performed over real user data from a variety of data sources
such as Facebook, Dropbox, and Gmail show that our approach significantly
improves search accuracy when compared with traditional search tools.
Pixel-Adaptive Convolutional Neural Networks
Authors: Hang Su, Varun Jampani, Deqing Sun, Orazio Gallo, Erik Learned-Miller, Jan Kautz
Date: 2019-04-10 00:00:00
Proceeding: CVPR 2019 6

Convolutions are the fundamental building block of CNNs. The fact that their
weights are spatially shared is one of the main reasons for their widespread
use, but it also is a major limitation, as it makes convolutions content
agnostic. We propose a pixel-adaptive convolution (PAC) operation, a simple yet
effective modification of standard convolutions, in which the filter weights
are multiplied with a spatially-varying kernel that depends on learnable, local
pixel features. PAC is a generalization of several popular filtering techniques
and thus can be used for a wide range of use cases. Specifically, we
demonstrate state-of-the-art performance when PAC is used for deep joint image
upsampling. PAC also offers an effective alternative to fully-connected CRF
(Full-CRF), called PAC-CRF, which performs competitively, while being
considerably faster. In addition, we also demonstrate that PAC can be used as a
drop-in replacement for convolution layers in pre-trained networks, resulting
in consistent performance improvements.
A Selective Overview of Deep Learning
Authors: Jianqing Fan, Cong Ma, Yiqiao Zhong
Date: 2019-04-10 00:00:00
Tasks: Deep Learning

Deep learning has arguably achieved tremendous success in recent years. In
simple words, deep learning uses the composition of many nonlinear functions to
model the complex dependency between input features and labels. While neural
networks have a long history, recent advances have greatly improved their
performance in computer vision, natural language processing, etc. From the
statistical and scientific perspective, it is natural to ask: What is deep
learning? What are the new characteristics of deep learning, compared with
classical methods? What are the theoretical foundations of deep learning? To
answer these questions, we introduce common neural network models (e.g.,
convolutional neural nets, recurrent neural nets, generative adversarial nets)
and training techniques (e.g., stochastic gradient descent, dropout, batch
normalization) from a statistical point of view. Along the way, we highlight
new characteristics of deep learning (including depth and over-parametrization)
and explain their practical and theoretical benefits. We also sample recent
results on theories of deep learning, many of which are only suggestive. While
a complete understanding of deep learning remains elusive, we hope that our
perspectives and discussions serve as a stimulus for new statistical research.