pwc-archive/papers-with-abstracts huggingface dataset:
Human-Centered Artificial Intelligence and Machine Learning
Authors: Mark O. Riedl
Date: 2019-01-31 00:00:00
Tasks: BIG-bench Machine Learning, Fairness

Humans are increasingly coming into contact with artificial intelligence and
machine learning systems. Human-centered artificial intelligence is a
perspective on AI and ML that algorithms must be designed with awareness that
they are part of a larger system consisting of humans. We lay forth an argument
that human-centered artificial intelligence can be broken down into two
aspects: (1) AI systems that understand humans from a sociocultural
perspective, and (2) AI systems that help humans understand them. We further
argue that issues of social responsibility such as fairness, accountability,
interpretability, and transparency.
Human Face Expressions from Images - 2D Face Geometry and 3D Face Local Motion versus Deep Neural Features
Authors: Rafal Pilarczyk, Xin Chang, Wladyslaw Skarbek
Date: 2019-01-31 00:00:00
Tasks: 3D geometry

Several computer algorithms for recognition of visible human emotions are
compared at the web camera scenario using CNN/MMOD face detector. The
recognition refers to four face expressions: smile, surprise, anger, and
neutral. At the feature extraction stage, the following three concepts of face
description are confronted: (a) static 2D face geometry represented by its 68
characteristic landmarks (FP68); (b) dynamic 3D geometry defined by motion
parameters for eight distinguished face parts (denoted as AU8) of personalized
Candide-3 model; (c) static 2D visual description as 2D array of gray scale
pixels (known as facial raw image). At the classification stage, the
performance of two major models are analyzed: (a) support vector machine (SVM)
with kernel options; (b) convolutional neural network (CNN) with variety of
relevant tensor processing layers and blocks of them. The models are trained
for frontal views of human faces while they are tested for arbitrary head
poses. For geometric features, the success rate (accuracy) indicate nearly
triple increase of performance of CNN with respect to SVM classifiers. For raw
images, CNN outperforms in accuracy its best geometric counterpart (AU/CNN) by
about 30 percent while the best SVM solutions are inferior nearly four times.
For F-score the high advantage of raw/CNN over geometric/CNN and geometric/SVM
is observed, as well. We conclude that contrary to CNN based emotion
classifiers, the generalization capability wrt human head pose is for SVM based
emotion classifiers poor.
Peer-to-peer Federated Learning on Graphs
Authors: Anusha Lalitha, Osman Cihan Kilinc, Tara Javidi, Farinaz Koushanfar
Date: 2019-01-31 00:00:00
Tasks: Federated Learning

We consider the problem of training a machine learning model over a network
of nodes in a fully decentralized framework. The nodes take a Bayesian-like
approach via the introduction of a belief over the model parameter space. We
propose a distributed learning algorithm in which nodes update their belief by
aggregate information from their one-hop neighbors to learn a model that best
fits the observations over the entire network. In addition, we also obtain
sufficient conditions to ensure that the probability of error is small for
every node in the network. We discuss approximations required for applying this
algorithm to train Deep Neural Networks (DNNs). Experiments on training linear
regression model and on training a DNN show that the proposed learning rule
algorithm provides a significant improvement in the accuracy compared to the
case where nodes learn without cooperation.
Bayesian nonparametric multiway regression for clustered binomial data
Authors: Eric F. Lock, Dipankar Bandyopadhyay
Date: 2019-01-31 00:00:00
Tasks: Clustering, regression

We introduce a Bayesian nonparametric regression model for data with multiway
(tensor) structure, motivated by an application to periodontal disease (PD)
data. Our outcome is the number of diseased sites measured over four different
tooth types for each subject, with subject-specific covariates available as
predictors. The outcomes are not well-characterized by simple parametric
models, so we use a nonparametric approach with a binomial likelihood wherein
the latent probabilities are drawn from a mixture with an arbitrary number of
components, analogous to a Dirichlet Process (DP). We use a flexible probit
stick-breaking formulation for the component weights that allows for covariate
dependence and clustering structure in the outcomes. The parameter space for
this model is large and multiway: patients $\times$ tooth types $\times$
covariates $\times$ components. We reduce its effective dimensionality, and
account for the multiway structure, via low-rank assumptions. We illustrate how
this can improve performance, and simplify interpretation, while still
providing sufficient flexibility. We describe a general and efficient Gibbs
sampling algorithm for posterior computation. The resulting fit to the PD data
outperforms competitors, and is interpretable and well-calibrated. An
interactive visual of the predictive model is available at
http://ericfrazerlock.com/toothdata/ToothDisplay.html , and the code is
available at https://github.com/lockEF/NonparametricMultiway .
Unsupervised Prediction of Negative Health Events Ahead of Time
Authors: Anahita Hosseini, Majid Sarrafzadeh
Date: 2019-01-31 00:00:00
Tasks: Anomaly Detection, Clustering, Online Clustering, Onset Detection, Time Series, Time Series Analysis

The emergence of continuous health monitoring and the availability of an
enormous amount of time series data has provided a great opportunity for the
advancement of personal health tracking. In recent years, unsupervised learning
methods have drawn special attention of researchers to tackle the sparse
annotation of health data and real-time detection of anomalies has been a
central problem of interest. However, one problem that has not been well
addressed before is the early prediction of forthcoming negative health events.
Early signs of an event can introduce subtle and gradual changes in the health
signal prior to its onset, detection of which can be invaluable in effective
prevention. In this study, we first demonstrate our observations on the
shortcoming of widely adopted anomaly detection methods in uncovering the
changes prior to a negative health event. We then propose a framework which
relies on online clustering of signal segment representations which are
automatically learned by a specially designed LSTM auto-encoder. We show the
effectiveness of our approach by predicting Bradycardia events in infants using
MIT-PICS dataset 1.3 minutes ahead of time with 68\% AUC score on average,
using no label supervision. Results of our study can indicate the viability of
our approach in the early detection of health events in other applications as
well.
A Generalized Language Model in Tensor Space
Authors: Lipeng Zhang, Peng Zhang, Xindian Ma, Shuqin Gu, Zhan Su, Dawei Song
Date: 2019-01-31 00:00:00
Tasks: Language Modeling, Language Modelling, model, Tensor Decomposition, Tensor Networks

In the literature, tensors have been effectively used for capturing the
context information in language models. However, the existing methods usually
adopt relatively-low order tensors, which have limited expressive power in
modeling language. Developing a higher-order tensor representation is
challenging, in terms of deriving an effective solution and showing its
generality. In this paper, we propose a language model named Tensor Space
Language Model (TSLM), by utilizing tensor networks and tensor decomposition.
In TSLM, we build a high-dimensional semantic space constructed by the tensor
product of word vectors. Theoretically, we prove that such tensor
representation is a generalization of the n-gram language model. We further
show that this high-order tensor representation can be decomposed to a
recursive calculation of conditional probability for language modeling. The
experimental results on Penn Tree Bank (PTB) dataset and WikiText benchmark
demonstrate the effectiveness of TSLM.
Spatial-Temporal Graph Convolutional Networks for Sign Language Recognition
Authors: Cleison Correia de Amorim, David Macêdo, Cleber Zanchettin
Date: 2019-01-31 00:00:00
Tasks: Sign Language Recognition

The recognition of sign language is a challenging task with an important role in society to facilitate the communication of deaf persons. We propose a new approach of Spatial-Temporal Graph Convolutional Network to sign language recognition based on the human skeletal movements. The method uses graphs to capture the signs dynamics in two dimensions, spatial and temporal, considering the complex aspects of the language. Additionally, we present a new dataset of human skeletons for sign language based on ASLLVD to contribute to future related studies.
Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images
Authors: Haozhe Xie, Hongxun Yao, Xiaoshuai Sun, Shangchen Zhou, Shengping Zhang
Date: 2019-01-31 00:00:00
Proceeding: ICCV 2019 10
Tasks: 3D Object Reconstruction, 3D Reconstruction, Decoder, Multi-View 3D Reconstruction

Recovering the 3D representation of an object from single-view or multi-view RGB images by deep neural networks has attracted increasing attention in the past few years. Several mainstream works (e.g., 3D-R2N2) use recurrent neural networks (RNNs) to fuse multiple feature maps extracted from input images sequentially. However, when given the same set of input images with different orders, RNN-based approaches are unable to produce consistent reconstruction results. Moreover, due to long-term memory loss, RNNs cannot fully exploit input images to refine reconstruction results. To solve these problems, we propose a novel framework for single-view and multi-view 3D reconstruction, named Pix2Vox. By using a well-designed encoder-decoder, it generates a coarse 3D volume from each input image. Then, a context-aware fusion module is introduced to adaptively select high-quality reconstructions for each part (e.g., table legs) from different coarse 3D volumes to obtain a fused 3D volume. Finally, a refiner further refines the fused 3D volume to generate the final output. Experimental results on the ShapeNet and Pix3D benchmarks indicate that the proposed Pix2Vox outperforms state-of-the-arts by a large margin. Furthermore, the proposed method is 24 times faster than 3D-R2N2 in terms of backward inference time. The experiments on ShapeNet unseen 3D categories have shown the superior generalization abilities of our method.
Distinguishing between Normal and Cancer Cells Using Autoencoder Node Saliency
Authors: Ya Ju Fan, Jonathan E. Allen, Sam Ade Jacobs, Brian C. Van Essen
Date: 2019-01-30 00:00:00
Tasks: Dimensionality Reduction

Gene expression profiles have been widely used to characterize patterns of
cellular responses to diseases. As data becomes available, scalable learning
toolkits become essential to processing large datasets using deep learning
models to model complex biological processes. We present an autoencoder to
capture nonlinear relationships recovered from gene expression profiles. The
autoencoder is a nonlinear dimension reduction technique using an artificial
neural network, which learns hidden representations of unlabeled data. We train
the autoencoder on a large collection of tumor samples from the National Cancer
Institute Genomic Data Commons, and obtain a generalized and unsupervised
latent representation. We leverage a HPC-focused deep learning toolkit,
Livermore Big Artificial Neural Network (LBANN) to efficiently parallelize the
training algorithm, reducing computation times from several hours to a few
minutes. With the trained autoencoder, we generate latent representations of a
small dataset, containing pairs of normal and cancer cells of various tumor
types. A novel measure called autoencoder node saliency (ANS) is introduced to
identify the hidden nodes that best differentiate various pairs of cells. We
compare our findings of the best classifying nodes with principal component
analysis and the visualization of t-distributed stochastic neighbor embedding.
We demonstrate that the autoencoder effectively extracts distinct gene features
for multiple learning tasks in the dataset.
Memory-Efficient Adaptive Optimization
Authors: Rohan Anil, Vineet Gupta, Tomer Koren, Yoram Singer
Date: 2019-01-30 00:00:00
Tasks: Language Modeling, Language Modelling, Machine Translation, Translation

Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling. However, these methods maintain second-order statistics for each parameter, thus introducing significant memory overheads that restrict the size of the model being used as well as the number of examples in a mini-batch. We describe an effective and flexible adaptive optimization method with greatly reduced memory overhead. Our method retains the benefits of per-parameter adaptivity while allowing significantly larger models and batch sizes. We give convergence guarantees for our method, and demonstrate its effectiveness in training very large translation and language models with up to 2-fold speedups compared to the state-of-the-art.
Which Factorization Machine Modeling is Better: A Theoretical Answer with Optimal Guarantee
Authors: Ming Lin, Shuang Qiu, Jieping Ye, Xiaomin Song, Qi Qian, Liang Sun, Shenghuo Zhu, Rong Jin
Date: 2019-01-30 00:00:00

Factorization machine (FM) is a popular machine learning model to capture the
second order feature interactions. The optimal learning guarantee of FM and its
generalized version is not yet developed. For a rank $k$ generalized FM of $d$
dimensional input, the previous best known sampling complexity is
$\mathcal{O}[k^{3}d\cdot\mathrm{polylog}(kd)]$ under Gaussian distribution.
This bound is sub-optimal comparing to the information theoretical lower bound
$\mathcal{O}(kd)$. In this work, we aim to tighten this bound towards optimal
and generalize the analysis to sub-gaussian distribution. We prove that when
the input data satisfies the so-called $\tau$-Moment Invertible Property, the
sampling complexity of generalized FM can be improved to
$\mathcal{O}[k^{2}d\cdot\mathrm{polylog}(kd)/\tau^{2}]$. When the second order
self-interaction terms are excluded in the generalized FM, the bound can be
improved to the optimal $\mathcal{O}[kd\cdot\mathrm{polylog}(kd)]$ up to the
logarithmic factors. Our analysis also suggests that the positive semi-definite
constraint in the conventional FM is redundant as it does not improve the
sampling complexity while making the model difficult to optimize. We evaluate
our improved FM model in real-time high precision GPS signal calibration task
to validate its superiority.
Natural Analysts in Adaptive Data Analysis
Authors: Tijana Zrnic, Moritz Hardt
Date: 2019-01-30 00:00:00
Tasks: Generalization Bounds

Adaptive data analysis is frequently criticized for its pessimistic generalization guarantees. The source of these pessimistic bounds is a model that permits arbitrary, possibly adversarial analysts that optimally use information to bias results. While being a central issue in the field, still lacking are notions of natural analysts that allow for more optimistic bounds faithful to the reality that typical analysts aren't adversarial. In this work, we propose notions of natural analysts that smoothly interpolate between the optimal non-adaptive bounds and the best-known adaptive generalization bounds. To accomplish this, we model the analyst's knowledge as evolving according to the rules of an unknown dynamical system that takes in revealed information and outputs new statistical queries to the data. This allows us to restrict the analyst through different natural control-theoretic notions. One such notion corresponds to a recency bias, formalizing an inability to arbitrarily use distant information. Another complementary notion formalizes an anchoring bias, a tendency to weight initial information more strongly. Both notions come with quantitative parameters that smoothly interpolate between the non-adaptive case and the fully adaptive case, allowing for a rich spectrum of intermediate analysts that are neither non-adaptive nor adversarial. Natural not only from a cognitive perspective, we show that our notions also capture standard optimization methods, like gradient descent in various settings. This gives a new interpretation to the fact that gradient descent tends to overfit much less than its adaptive nature might suggest.
On the Consistency of Top-k Surrogate Losses
Authors: Forest Yang, Sanmi Koyejo
Date: 2019-01-30 00:00:00
Proceeding: ICML 2020 1
Tasks: General Classification

The top-$k$ error is often employed to evaluate performance for challenging classification tasks in computer vision as it is designed to compensate for ambiguity in ground truth labels. This practical success motivates our theoretical analysis of consistent top-$k$ classification. Surprisingly, it is not rigorously understood when taking the $k$-argmax of a vector is guaranteed to return the $k$-argmax of another vector, though doing so is crucial to describe Bayes optimality; we do both tasks. Then, we define top-$k$ calibration and show it is necessary and sufficient for consistency. Based on the top-$k$ calibration analysis, we propose a class of top-$k$ calibrated Bregman divergence surrogates. Our analysis continues by showing previously proposed hinge-like top-$k$ surrogate losses are not top-$k$ calibrated and suggests no convex hinge loss is top-$k$ calibrated. On the other hand, we propose a new hinge loss which is consistent. We explore further, showing our hinge loss remains consistent under a restriction to linear functions, while cross entropy does not. Finally, we exhibit a differentiable, convex loss function which is top-$k$ calibrated for specific $k$.
Emerging Convolutions for Generative Normalizing Flows
Authors: Emiel Hoogeboom, Rianne van den Berg, Max Welling
Date: 2019-01-30 00:00:00
Tasks: Image Generation

Generative flows are attractive because they admit exact likelihood optimization and efficient image synthesis. Recently, Kingma & Dhariwal (2018) demonstrated with Glow that generative flows are capable of generating high quality images. We generalize the 1 x 1 convolutions proposed in Glow to invertible d x d convolutions, which are more flexible since they operate on both channel and spatial axes. We propose two methods to produce invertible convolutions that have receptive fields identical to standard convolutions: Emerging convolutions are obtained by chaining specific autoregressive convolutions, and periodic convolutions are decoupled in the frequency domain. Our experiments show that the flexibility of d x d convolutions significantly improves the performance of generative flow models on galaxy images, CIFAR10 and ImageNet.
Which Surrogate Works for Empirical Performance Modelling? A Case Study with Differential Evolution
Authors: Ke Li, Zilin Xiang, Kay Chen Tan
Date: 2019-01-30 00:00:00
Tasks: regression

It is not uncommon that meta-heuristic algorithms contain some intrinsic
parameters, the optimal configuration of which is crucial for achieving their
peak performance. However, evaluating the effectiveness of a configuration is
expensive, as it involves many costly runs of the target algorithm. Perhaps
surprisingly, it is possible to build a cheap-to-evaluate surrogate that models
the algorithm's empirical performance as a function of its parameters. Such
surrogates constitute an important building block for understanding algorithm
performance, algorithm portfolio/selection, and the automatic algorithm
configuration. In principle, many off-the-shelf machine learning techniques can
be used to build surrogates. In this paper, we take the differential evolution
(DE) as the baseline algorithm for proof-of-concept study. Regression models
are trained to model the DE's empirical performance given a parameter
configuration. In particular, we evaluate and compare four popular regression
algorithms both in terms of how well they predict the empirical performance
with respect to a particular parameter configuration, and also how well they
approximate the parameter versus the empirical performance landscapes.
The Evolved Transformer
Authors: David R. So, Chen Liang, Quoc V. Le
Date: 2019-01-30 00:00:00
Tasks: Machine Translation, Neural Architecture Search

Recent works have highlighted the strength of the Transformer architecture on sequence tasks while, at the same time, neural architecture search (NAS) has begun to outperform human-designed models. Our goal is to apply NAS to search for a better alternative to the Transformer. We first construct a large search space inspired by the recent advances in feed-forward sequence models and then run evolutionary architecture search with warm starting by seeding our initial population with the Transformer. To directly search on the computationally expensive WMT 2014 English-German translation task, we develop the Progressive Dynamic Hurdles method, which allows us to dynamically allocate more resources to more promising candidate models. The architecture found in our experiments -- the Evolved Transformer -- demonstrates consistent improvement over the Transformer on four well-established language tasks: WMT 2014 English-German, WMT 2014 English-French, WMT 2014 English-Czech and LM1B. At a big model size, the Evolved Transformer establishes a new state-of-the-art BLEU score of 29.8 on WMT'14 English-German; at smaller sizes, it achieves the same quality as the original "big" Transformer with 37.6% less parameters and outperforms the Transformer by 0.7 BLEU at a mobile-friendly model size of 7M parameters.
Code Farming: A Process for Creating Generic Computational Building Blocks
Authors: David Landaeta
Date: 2019-01-30 00:00:00

Motivated by a desire to improve on the current state of the art in genetic
programming, and aided by recent progress in understanding the computational
aspects of evolutionary systems, we describe a process that creates a set of
generic computational building blocks for the purpose of seeding initial
populations of programs in any genetic programming system. This provides an
advantage over the standard approach of initializing the population purely
randomly in that it avoids the need to constantly rediscover such building
blocks. It is also better than seeding the initial population with hand-coded
building blocks, since it lessens the amount of human intervention required by
the system.