pwc-archive/papers-with-abstracts huggingface dataset:
Fair is Better than Sensational:Man is to Doctor as Woman is to Doctor
Authors: Malvina Nissim, Rik van Noord, Rob van der Goot
Date: 2019-05-23 00:00:00
Tasks: Bias Detection, Diagnostic, Word Embeddings

Analogies such as "man is to king as woman is to X" are often used to illustrate the amazing power of word embeddings. Concurrently, they have also been used to expose how strongly human biases are encoded in vector spaces built on natural language, like "man is to computer programmer as woman is to homemaker". Recent work has shown that analogies are in fact not such a diagnostic for bias, and other methods have been proven to be more apt to the task. However, beside the intrinsic problems with the analogy task as a bias detection tool, in this paper we show that a series of issues related to how analogies have been implemented and used might have yielded a distorted picture of bias in word embeddings. Human biases are present in word embeddings and need to be addressed. Analogies, though, are probably not the right tool to do so. Also, the way they have been most often used has exacerbated some possibly non-existing biases and perhaps hid others. Because they are still widely popular, and some of them have become classics within and outside the NLP community, we deem it important to provide a series of clarifications that should put well-known, and potentially new cases into the right perspective.
Interpreting a Recurrent Neural Network's Predictions of ICU Mortality Risk
Authors: Long V. Ho, Melissa D. Aczon, David Ledbetter, Randall Wetzel
Date: 2019-05-23 00:00:00
Tasks: Feature Importance, ICU Mortality, Time Series Analysis

Deep learning has demonstrated success in many applications; however, their use in healthcare has been limited due to the lack of transparency into how they generate predictions. Algorithms such as Recurrent Neural Networks (RNNs) when applied to Electronic Medical Records (EMR) introduce additional barriers to transparency because of the sequential processing of the RNN and the multi-modal nature of EMR data. This work seeks to improve transparency by: 1) introducing Learned Binary Masks (LBM) as a method for identifying which EMR variables contributed to an RNN model's risk of mortality (ROM) predictions for critically ill children; and 2) applying KernelSHAP for the same purpose. Given an individual patient, LBM and KernelSHAP both generate an attribution matrix that shows the contribution of each input feature to the RNN's sequence of predictions for that patient. Attribution matrices can be aggregated in many ways to facilitate different levels of analysis of the RNN model and its predictions. Presented are three methods of aggregations and analyses: 1) over volatile time periods within individual patient predictions, 2) over populations of ICU patients sharing specific diagnoses, and 3) across the general population of critically ill children.
Why Didn't You Listen to Me? Comparing User Control of Human-in-the-Loop Topic Models
Authors: Varun Kumar, Alison Smith-Renner, Leah Findlater, Kevin Seppi, Jordan Boyd-Graber
Date: 2019-05-23 00:00:00
Proceeding: ACL 2019 7
Tasks: Topic Models

To address the lack of comparative evaluation of Human-in-the-Loop Topic Modeling (HLTM) systems, we implement and evaluate three contrasting HLTM modeling approaches using simulation experiments. These approaches extend previously proposed frameworks, including constraints and informed prior-based methods. Users should have a sense of control in HLTM systems, so we propose a control metric to measure whether refinement operations' results match users' expectations. Informed prior-based methods provide better control than constraints, but constraints yield higher quality topics.
Accelerating Langevin Sampling with Birth-death
Authors: Yulong Lu, Jianfeng Lu, James Nolen
Date: 2019-05-23 00:00:00
Tasks: Bayesian Inference

A fundamental problem in Bayesian inference and statistical machine learning is to efficiently sample from multimodal distributions. Due to metastability, multimodal distributions are difficult to sample using standard Markov chain Monte Carlo methods. We propose a new sampling algorithm based on a birth-death mechanism to accelerate the mixing of Langevin diffusion. Our algorithm is motivated by its mean field partial differential equation (PDE), which is a Fokker-Planck equation supplemented by a nonlocal birth-death term. This PDE can be viewed as a gradient flow of the Kullback-Leibler divergence with respect to the Wasserstein-Fisher-Rao metric. We prove that under some assumptions the asymptotic convergence rate of the nonlocal PDE is independent of the potential barrier, in contrast to the exponential dependence in the case of the Langevin diffusion. We illustrate the efficiency of the birth-death accelerated Langevin method through several analytical examples and numerical experiments.
Copy this Sentence
Authors: Vasileios Lioutas, Andriy Drozdyuk
Date: 2019-05-23 00:00:00
Tasks: Sentence

Attention is an operation that selects some largest element from some set, where the notion of largest is defined elsewhere. Applying this operation to sequence to sequence mapping results in significant improvements to the task at hand. In this paper we provide the mathematical definition of attention and examine its application to sequence to sequence models. We highlight the exact correspondences between machine learning implementations of attention and our mathematical definition. We provide clear evidence of effectiveness of attention mechanisms evaluating models with varying degrees of attention on a very simple task: copying a sentence. We find that models that make greater use of attention perform much better on sequence to sequence mapping tasks, converge faster and are more stable.
Distributional Policy Optimization: An Alternative Approach for Continuous Control
Authors: Chen Tessler, Guy Tennenholtz, Shie Mannor
Date: 2019-05-23 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: continuous-control, Continuous Control, Policy Gradient Methods

We identify a fundamental problem in policy gradient-based methods in continuous control. As policy gradient methods require the agent's underlying probability distribution, they limit policy representation to parametric distribution classes. We show that optimizing over such sets results in local movement in the action space and thus convergence to sub-optimal solutions. We suggest a novel distributional framework, able to represent arbitrary distribution functions over the continuous action space. Using this framework, we construct a generative scheme, trained using an off-policy actor-critic paradigm, which we call the Generative Actor Critic (GAC). Compared to policy gradient methods, GAC does not require knowledge of the underlying probability distribution, thereby overcoming these limitations. Empirical evaluation shows that our approach is comparable and often surpasses current state-of-the-art baselines in continuous domains.
Computationally Efficient Feature Significance and Importance for Machine Learning Models
Authors: Enguerrand Horel, Kay Giesecke
Date: 2019-05-23 00:00:00
Tasks: BIG-bench Machine Learning, Feature Importance

We develop a simple and computationally efficient significance test for the features of a machine learning model. Our forward-selection approach applies to any model specification, learning task and variable type. The test is non-asymptotic, straightforward to implement, and does not require model refitting. It identifies the statistically significant features as well as feature interactions of any order in a hierarchical manner, and generates a model-free notion of feature importance. Experimental and empirical results illustrate its performance.
Plane-Based Optimization of Geometry and Texture for RGB-D Reconstruction of Indoor Scenes
Authors: Chao Wang, Xiaohu Guo
Date: 2019-05-23 00:00:00
Tasks: 3D Reconstruction, RGB-D Reconstruction

We present a novel approach to reconstruct RGB-D indoor scene with plane primitives. Our approach takes as input a RGB-D sequence and a dense coarse mesh reconstructed by some 3D reconstruction method on the sequence, and generate a lightweight, low-polygonal mesh with clear face textures and sharp features without losing geometry details from the original scene. To achieve this, we firstly partition the input mesh with plane primitives, simplify it into a lightweight mesh next, then optimize plane parameters, camera poses and texture colors to maximize the photometric consistency across frames, and finally optimize mesh geometry to maximize consistency between geometry and planes. Compared to existing planar reconstruction methods which only cover large planar regions in the scene, our method builds the entire scene by adaptive planes without losing geometry details and preserves sharp features in the final mesh. We demonstrate the effectiveness of our approach by applying it onto several RGB-D scans and comparing it to other state-of-the-art reconstruction methods.
A model of brain morphological changes related to aging and Alzheimer's disease from cross-sectional assessments
Authors: Raphaël Sivera, Hervé Delingette, Marco Lorenzi, Xavier Pennec, Nicholas Ayache
Date: 2019-05-23 00:00:00

In this study we propose a deformation-based framework to jointly model the influence of aging and Alzheimer's disease (AD) on the brain morphological evolution. Our approach combines a spatio-temporal description of both processes into a generative model. A reference morphology is deformed along specific trajectories to match subject specific morphologies. It is used to define two imaging progression markers: 1) a morphological age and 2) a disease score. These markers can be computed locally in any brain region. The approach is evaluated on brain structural magnetic resonance images (MRI) from the ADNI database. The generative model is first estimated on a control population, then, for each subject, the markers are computed for each acquisition. The longitudinal evolution of these markers is then studied in relation with the clinical diagnosis of the subjects and used to generate possible morphological evolution. In the model, the morphological changes associated with normal aging are mainly found around the ventricles, while the Alzheimer's disease specific changes are more located in the temporal lobe and the hippocampal area. The statistical analysis of these markers highlights differences between clinical conditions even though the inter-subject variability is quiet high. In this context, the model can be used to generate plausible morphological trajectories associated with the disease. Our method gives two interpretable scalar imaging biomarkers assessing the effects of aging and disease on brain morphology at the individual and population level. These markers confirm an acceleration of apparent aging for Alzheimer's subjects and can help discriminate clinical conditions even in prodromal stages. More generally, the joint modeling of normal and pathological evolutions shows promising results to describe age-related brain diseases over long time scales.
Randomized Reference Classifier with Gaussian Distribution and Soft Confusion Matrix Applied to the Improving Weak Classifiers
Authors: Pawel Trajdos, Marek Kurzynski
Date: 2019-05-23 00:00:00

In this paper, an issue of building the RRC model using probability distributions other than beta distribution is addressed. More precisely, in this paper, we propose to build the RRR model using the truncated normal distribution. Heuristic procedures for expected value and the variance of the truncated-normal distribution are also proposed. The proposed approach is tested using SCM-based model for testing the consequences of applying the truncated normal distribution in the RRC model. The experimental evaluation is performed using four different base classifiers and seven quality measures. The results showed that the proposed approach is comparable to the RRC model built using beta distribution. What is more, for some base classifiers, the truncated-normal-based SCM algorithm turned out to be better at discovering objects coming from minority classes.
Patent Citation Dynamics Modeling via Multi-Attention Recurrent Networks
Authors: Taoran Ji, Zhiqian Chen, Nathan Self, Kaiqun Fu, Chang-Tien Lu, Naren Ramakrishnan
Date: 2019-05-22 00:00:00
Tasks: Citation Prediction, Point Processes

Modeling and forecasting forward citations to a patent is a central task for the discovery of emerging technologies and for measuring the pulse of inventive progress. Conventional methods for forecasting these forward citations cast the problem as analysis of temporal point processes which rely on the conditional intensity of previously received citations. Recent approaches model the conditional intensity as a chain of recurrent neural networks to capture memory dependency in hopes of reducing the restrictions of the parametric form of the intensity function. For the problem of patent citations, we observe that forecasting a patent's chain of citations benefits from not only the patent's history itself but also from the historical citations of assignees and inventors associated with that patent. In this paper, we propose a sequence-to-sequence model which employs an attention-of-attention mechanism to capture the dependencies of these multiple time sequences. Furthermore, the proposed model is able to forecast both the timestamp and the category of a patent's next citation. Extensive experiments on a large patent citation dataset collected from USPTO demonstrate that the proposed model outperforms state-of-the-art models at forward citation forecasting.
Scaling Up Collaborative Filtering Data Sets through Randomized Fractal Expansions
Authors: Francois Belletti, Karthik Lakshmanan, Walid Krichene, Nicolas Mayoraz, Yi-fan Chen, John Anderson, Taylor Robie, Tayo Oguntebi, Dan Shirron, Amit Bleiwess
Date: 2019-04-08 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Recommender system research suffers from a disconnect between the size of
academic data sets and the scale of industrial production systems. In order to
bridge that gap, we propose to generate large-scale user/item interaction data
sets by expanding pre-existing public data sets. Our key contribution is a
technique that expands user/item incidence matrices matrices to large numbers
of rows (users), columns (items), and non-zero values (interactions). The
proposed method adapts Kronecker Graph Theory to preserve key higher order
statistical properties such as the fat-tailed distribution of user engagements,
item popularity, and singular value spectra of user/item interaction matrices.
Preserving such properties is key to building large realistic synthetic data
sets which in turn can be employed reliably to benchmark recommender systems
and the systems employed to train them. We further apply our stochastic
expansion algorithm to the binarized MovieLens 20M data set, which comprises
20M interactions between 27K movies and 138K users. The resulting expanded data
set has 1.2B ratings, 2.2M users, and 855K items, which can be scaled up or
down.
Unsupervised Machine Learning for the Discovery of Latent Disease Clusters and Patient Subgroups Using Electronic Health Records
Authors: Yanshan Wang, Yiqing Zhao, Terry M. Therneau, Elizabeth J. Atkinson, Ahmad P. Tafti, Nan Zhang, Shreyasee Amin, Andrew H. Limper, Hongfang Liu
Date: 2019-05-17 00:00:00
Tasks: BIG-bench Machine Learning, Epidemiology, Survival Analysis

Machine learning has become ubiquitous and a key technology on mining electronic health records (EHRs) for facilitating clinical research and practice. Unsupervised machine learning, as opposed to supervised learning, has shown promise in identifying novel patterns and relations from EHRs without using human created labels. In this paper, we investigate the application of unsupervised machine learning models in discovering latent disease clusters and patient subgroups based on EHRs. We utilized Latent Dirichlet Allocation (LDA), a generative probabilistic model, and proposed a novel model named Poisson Dirichlet Model (PDM), which extends the LDA approach using a Poisson distribution to model patients' disease diagnoses and to alleviate age and sex factors by considering both observed and expected observations. In the empirical experiments, we evaluated LDA and PDM on three patient cohorts with EHR data retrieved from the Rochester Epidemiology Project (REP), for the discovery of latent disease clusters and patient subgroups. We compared the effectiveness of LDA and PDM in identifying latent disease clusters through the visualization of disease representations learned by two approaches. We also tested the performance of LDA and PDM in differentiating patient subgroups through survival analysis, as well as statistical analysis. The experimental results show that the proposed PDM could effectively identify distinguished disease clusters by alleviating the impact of age and sex, and that LDA could stratify patients into more differentiable subgroups than PDM in terms of p-values. However, the subgroups discovered by PDM might imply the underlying patterns of diseases of greater interest in epidemiology research due to the alleviation of age and sex. Both unsupervised machine learning approaches could be leveraged to discover patient subgroups using EHRs but with different foci.
Unsupervised Community Detection with Modularity-Based Attention Model
Authors: Ivan Lobov, Sergey Ivanov
Date: 2019-05-20 00:00:00
Tasks: Clustering, Community Detection, Graph Neural Network, model

In this paper we take a problem of unsupervised nodes clustering on graphs and show how recent advances in attention models can be applied successfully in a "hard" regime of the problem. We propose an unsupervised algorithm that encodes Bethe Hessian embeddings by optimizing soft modularity loss and argue that our model is competitive to both classical and Graph Neural Network (GNN) models while it can be trained on a single graph.
Predicting Brazilian court decisions
Authors: André Lage-Freitas, Héctor Allende-Cid, Orivaldo Santana, Lívia de Oliveira-Lage
Date: 2019-04-20 00:00:00

Predicting case outcomes is useful but still an extremely hard task for
attorneys and other Law professionals. It is not easy to search case
information to extract valuable information as this requires dealing with huge
data sets and their complexity. For instance, the complexity of Brazil legal
system along with the high litigation rates makes this problem even harder.
This paper introduces an approach for predicting Brazilian court decisions
which is also able to predict whether the decision will be unanimous. We
developed a working prototype which performs 79% of accuracy (F1-score) on a
data set composed of 4,043 cases from a Brazilian court. To our knowledge, this
is the first study to forecast judge decisions in Brazil.
OptStream: Releasing Time Series Privately
Authors: Ferdinando Fioretto, Pascal Van Hentenryck
Date: 2018-08-06 00:00:00
Tasks: Decision Making, Load Forecasting, Time Series, Time Series Analysis

Many applications of machine learning and optimization operate on data
streams. While these datasets are fundamental to fuel decision-making
algorithms, often they contain sensitive information about individuals and
their usage poses significant privacy risks. Motivated by an application in
energy systems, this paper presents OPTSTREAM, a novel algorithm for releasing
differentially private data streams under the w-event model of privacy.
OPTSTREAM is a 4-step procedure consisting of sampling, perturbation,
reconstruction, and post-processing modules. First, the sampling module selects
a small set of points to access in each period of interest. Then, the
perturbation module adds noise to the sampled data points to guarantee privacy.
Next, the reconstruction module reassembles non-sampled data points from the
perturbed sample points. Finally, the post-processing module uses convex
optimization over the private output of the previous modules, as well as the
private answers of additional queries on the data stream, to improve accuracy
by redistributing the added noise. OPTSTREAM is evaluated on a test case
involving the release of a real data stream from the largest European
transmission operator. Experimental results show that OPTSTREAM may not only
improve the accuracy of state-of-the-art methods by at least one order of
magnitude but also supports accurate load forecasting on the private data.
Learning a model of facial shape and expression from 4D scans
Authors: Tianye Li, Timo Bolkart, Michael J. Black, Hao Li, Javier Romero
Date: 2017-11-01 00:00:00
Proceeding: SIGGRAPH Asia 2017 11
Tasks: 3D Face Animation, 3D Face Modelling, 3D Face Reconstruction, Face Alignment, Face Generation, Face Model

he field of 3D face modeling has a large gap between high-end and low-end methods. At the high end, the best facial animation is indistinguishablefrom real humans, but this comes at the cost of extensive manual labor. At the low end, face capture from consumer depth sensors relies on 3D face models that are not expressive enough to capture the variability in natural facial shape and expression. We seek a middle ground by learning a facial model from thousands of accurately aligned 3D scans. Our FLAME model (Faces Learned with an Articulated Model and Expressions) is designed to work with existing graphics software and be easy to fit to data. FLAME uses a linear shape space trained from 3800 scans of human heads. FLAME combines this linear shape space with an articulated jaw, neck, and eyeballs, pose-dependent corrective blendshapes, and additional global expression blendshapes. The pose and expression dependent articulations are learned from 4D face sequences in the D3DFACS dataset along with additional 4D sequences. We accurately register a template mesh to the scan sequences and make the D3DFACS registrations available for research purposes. In total the model is trained from over 33, 000 scans. FLAME is low-dimensional but more expressive than the FaceWarehouse model and the Basel Face Model. We compare FLAME to these models by fitting them to static 3D scans and 4D sequences using the same optimization method. FLAME is significantly more accurate and is available for research purposes.
Fair Resource Allocation in Federated Learning
Authors: Tian Li, Maziar Sanjabi, Ahmad Beirami, Virginia Smith
Date: 2019-05-25 00:00:00
Proceeding: ICLR 2020 1
Tasks: Fairness, Federated Learning

Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair (specifically, a more uniform) accuracy distribution across devices in federated networks. To solve q-FFL, we devise a communication-efficient method, q-FedAvg, that is suited to federated networks. We validate both the effectiveness of q-FFL and the efficiency of q-FedAvg on a suite of federated datasets with both convex and non-convex models, and show that q-FFL (along with q-FedAvg) outperforms existing baselines in terms of the resulting fairness, flexibility, and efficiency.