pwc-archive/papers-with-abstracts huggingface dataset:
DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs
Authors: Ali Sadeghian, Mohammadreza Armandpour, Patrick Ding, Daisy Zhe Wang
Date: 2019-10-31 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Inductive knowledge graph completion, Inductive Link Prediction, Knowledge Graphs, Link Prediction, Prediction

In this paper, we study the problem of learning probabilistic logical rules for inductive and interpretable link prediction. Despite the importance of inductive link prediction, most previous works focused on transductive link prediction and cannot manage previously unseen entities. Moreover, they are black-box models that are not easily explainable for humans. We propose DRUM, a scalable and differentiable approach for mining first-order logical rules from knowledge graphs which resolves these problems. We motivate our method by making a connection between learning confidence scores for each rule and low-rank tensor approximation. DRUM uses bidirectional RNNs to share useful information across the tasks of learning rules for different relations. We also empirically demonstrate the efficiency of DRUM over existing rule mining methods for inductive link prediction on a variety of benchmark datasets.
Context-Aware Local Differential Privacy
Authors: Jayadev Acharya, Keith Bonawitz, Peter Kairouz, Daniel Ramage, Ziteng Sun
Date: 2019-10-31 00:00:00

Local differential privacy (LDP) is a strong notion of privacy for individual users that often comes at the expense of a significant drop in utility. The classical definition of LDP assumes that all elements in the data domain are equally sensitive. However, in many applications, some symbols are more sensitive than others. This work proposes a context-aware framework of local differential privacy that allows a privacy designer to incorporate the application's context into the privacy definition. For binary data domains, we provide a universally optimal privatization scheme and highlight its connections to Warner's randomized response (RR) and Mangat's improved response. Motivated by geolocation and web search applications, for $k$-ary data domains, we consider two special cases of context-aware LDP: block-structured LDP and high-low LDP. We study discrete distribution estimation and provide communication-efficient, sample-optimal schemes and information-theoretic lower bounds for both models. We show that using contextual information can require fewer samples than classical LDP to achieve the same accuracy.
Deep learning assessment of breast terminal duct lobular unit involution: towards automated prediction of breast cancer risk
Authors: Suzanne C. Wetstein, Allison M Onken, Christina Luffman, Gabrielle M Baker, Michael E Pyle, Kevin H Kensler, Ying Liu, Bart Bakker, Ruud Vlutters, Marinus B van Leeuwen, Laura C Collins, Stuart J Schnitt, Josien PW Pluim, Rulla M. Tamimi, Yujing J. Heng, Mitko Veta
Date: 2019-10-31 00:00:00
Tasks: whole slide images

Terminal ductal lobular unit (TDLU) involution is the regression of milk-producing structures in the breast. Women with less TDLU involution are more likely to develop breast cancer. A major bottleneck in studying TDLU involution in large cohort studies is the need for labor-intensive manual assessment of TDLUs. We developed a computational pathology solution to automatically capture TDLU involution measures. Whole slide images (WSIs) of benign breast biopsies were obtained from the Nurses' Health Study (NHS). A first set of 92 WSIs was annotated for TDLUs, acini and adipose tissue to train deep convolutional neural network (CNN) models for detection of acini, and segmentation of TDLUs and adipose tissue. These networks were integrated into a single computational method to capture TDLU involution measures including number of TDLUs per tissue area, median TDLU span and median number of acini per TDLU. We validated our method on 40 additional WSIs by comparing with manually acquired measures. Our CNN models detected acini with an F1 score of 0.73$\pm$0.09, and segmented TDLUs and adipose tissue with Dice scores of 0.86$\pm$0.11 and 0.86$\pm$0.04, respectively. The inter-observer ICC scores for manual assessments on 40 WSIs of number of TDLUs per tissue area, median TDLU span, and median acini count per TDLU were 0.71, 95% CI [0.51, 0.83], 0.81, 95% CI [0.67, 0.90], and 0.73, 95% CI [0.54, 0.85], respectively. Intra-observer reliability was evaluated on 10/40 WSIs with ICC scores of >0.8. Inter-observer ICC scores between automated results and the mean of the two observers were: 0.80, 95% CI [0.63, 0.90] for number of TDLUs per tissue area, 0.57, 95% CI [0.19, 0.77] for median TDLU span, and 0.80, 95% CI [0.62, 0.89] for median acini count per TDLU. TDLU involution measures evaluated by manual and automated assessment were inversely associated with age and menopausal status.
Modeling Feature Representations for Affective Speech using Generative Adversarial Networks
Authors: Saurabh Sahu, Rahul Gupta, Carol Espy-Wilson
Date: 2019-10-31 00:00:00
Tasks: Cross-corpus, Emotion Recognition, Natural Language Understanding

Emotion recognition is a classic field of research with a typical setup extracting features and feeding them through a classifier for prediction. On the other hand, generative models jointly capture the distributional relationship between emotions and the feature profiles. Relatively recently, Generative Adversarial Networks (GANs) have surfaced as a new class of generative models and have shown considerable success in modeling distributions in the fields of computer vision and natural language understanding. In this work, we experiment with variants of GAN architectures to generate feature vectors corresponding to an emotion in two ways: (i) A generator is trained with samples from a mixture prior. Each mixture component corresponds to an emotional class and can be sampled to generate features from the corresponding emotion. (ii) A one-hot vector corresponding to an emotion can be explicitly used to generate the features. We perform analysis on such models and also propose different metrics used to measure the performance of the GAN models in their ability to generate realistic synthetic samples. Apart from evaluation on a given dataset of interest, we perform a cross-corpus study where we study the utility of the synthetic samples as additional training data in low resource conditions.
Chirality Nets for Human Pose Regression
Authors: Raymond A. Yeh, Yuan-Ting Hu, Alexander G. Schwing
Date: 2019-10-31 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: 3D Human Pose Estimation, 3D Pose Estimation, Activity Recognition, Pose Estimation, regression, Skeleton Based Action Recognition

We propose Chirality Nets, a family of deep nets that is equivariant to the "chirality transform," i.e., the transformation to create a chiral pair. Through parameter sharing, odd and even symmetry, we propose and prove variants of standard building blocks of deep nets that satisfy the equivariance property, including fully connected layers, convolutional layers, batch-normalization, and LSTM/GRU cells. The proposed layers lead to a more data efficient representation and a reduction in computation by exploiting symmetry. We evaluate chirality nets on the task of human pose regression, which naturally exploits the left/right mirroring of the human body. We study three pose regression tasks: 3D pose estimation from video, 2D pose forecasting, and skeleton based activity recognition. Our approach achieves/matches state-of-the-art results, with more significant gains on small datasets and limited-data settings.
PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learning
Authors: Iou-Jen Liu, Raymond A. Yeh, Alexander G. Schwing
Date: 2019-10-31 00:00:00
Tasks: Deep Reinforcement Learning, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Sample efficiency and scalability to a large number of agents are two important goals for multi-agent reinforcement learning systems. Recent works got us closer to those goals, addressing non-stationarity of the environment from a single agent's perspective by utilizing a deep net critic which depends on all observations and actions. The critic input concatenates agent observations and actions in a user-specified order. However, since deep nets aren't permutation invariant, a permuted input changes the critic output despite the environment remaining identical. To avoid this inefficiency, we propose a 'permutation invariant critic' (PIC), which yields identical output irrespective of the agent permutation. This consistent representation enables our model to scale to 30 times more agents and to achieve improvements of test episode reward between 15% to 50% on the challenging multi-agent particle environment (MPE).
Does deep learning always outperform simple linear regression in optical imaging?
Authors: Shuming Jiao, Yang Gao, Jun Feng, Ting Lei, Xiaocong Yuan
Date: 2019-10-31 00:00:00
Tasks: Deep Learning, regression

Deep learning has been extensively applied in many optical imaging applications in recent years. Despite the success, the limitations and drawbacks of deep learning in optical imaging have been seldom investigated. In this work, we show that conventional linear-regression-based methods can outperform the previously proposed deep learning approaches for two black-box optical imaging problems in some extent. Deep learning demonstrates its weakness especially when the number of training samples is small. The advantages and disadvantages of linear-regression-based methods and deep learning are analyzed and compared. Since many optical systems are essentially linear, a deep learning network containing many nonlinearity functions sometimes may not be the most suitable option.
Continual Multi-task Gaussian Processes
Authors: Pablo Moreno-Muñoz, Antonio Artés-Rodríguez, Mauricio A. Álvarez
Date: 2019-10-31 00:00:00
Tasks: Bayesian Inference, Continual Learning, Gaussian Processes, Stochastic Optimization, Variational Inference

We address the problem of continual learning in multi-task Gaussian process (GP) models for handling sequential input-output observations. Our approach extends the existing prior-posterior recursion of online Bayesian inference, i.e.\ past posterior discoveries become future prior beliefs, to the infinite functional space setting of GP. For a reason of scalability, we introduce variational inference together with an sparse approximation based on inducing inputs. As a consequence, we obtain tractable continual lower-bounds where two novel Kullback-Leibler (KL) divergences intervene in a natural way. The key technical property of our method is the recursive reconstruction of conditional GP priors conditioned on the variational parameters learned so far. To achieve this goal, we introduce a novel factorization of past variational distributions, where the predictive GP equation propagates the posterior uncertainty forward. We then demonstrate that it is possible to derive GP models over many types of sequential observations, either discrete or continuous and amenable to stochastic optimization. The continual inference approach is also applicable to scenarios where potential multi-channel or heterogeneous observations might appear. Extensive experiments demonstrate that the method is fully scalable, shows a reliable performance and is robust to uncertainty error propagation over a plenty of synthetic and real-world datasets.
Hierarchical Expert Networks for Meta-Learning
Authors: Heinke Hihn, Daniel A. Braun
Date: 2019-10-31 00:00:00
Proceeding: ICML Workshop LifelongML 2020 7
Tasks: image-classification, Image Classification, Meta-Learning, regression, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The goal of meta-learning is to train a model on a variety of learning tasks, such that it can adapt to new problems within only a few iterations. Here we propose a principled information-theoretic model that optimally partitions the underlying problem space such that specialized expert decision-makers solve the resulting sub-problems. To drive this specialization we impose the same kind of information processing constraints both on the partitioning and the expert decision-makers. We argue that this specialization leads to efficient adaptation to new tasks. To demonstrate the generality of our approach we evaluate three meta-learning domains: image classification, regression, and reinforcement learning.
Causal Inference via Conditional Kolmogorov Complexity using MDL Binning
Authors: Daniel Goldfarb, Scott Evans
Date: 2019-10-31 00:00:00
Tasks: Causal Inference

Recent developments have linked causal inference with Algorithmic Information Theory, and methods have been developed that utilize Conditional Kolmogorov Complexity to determine causation between two random variables. We present a method for inferring causal direction between continuous variables by using an MDL Binning technique for data discretization and complexity calculation. Our method captures the shape of the data and uses it to determine which variable has more information about the other. Its high predictive performance and robustness is shown on several real world use cases.
Sparsely Activated Networks: A new method for decomposing and compressing data
Authors: Paschalis Bizopoulos
Date: 2019-10-30 00:00:00

Recent literature on unsupervised learning focused on designing structural priors with the aim of learning meaningful features, but without considering the description length of the representations. In this thesis, first we introduce the $\varphi$ metric that evaluates unsupervised models based on their reconstruction accuracy and the degree of compression of their internal representations. We then present and define two activation functions (Identity, ReLU) as base of reference and three sparse activation functions (top-k absolutes, Extrema-Pool indices, Extrema) as candidate structures that minimize the previously defined metric $\varphi$. We lastly present Sparsely Activated Networks (SANs) that consist of kernels with shared weights that, during encoding, are convolved with the input and then passed through a sparse activation function. During decoding, the same weights are convolved with the sparse activation map and subsequently the partial reconstructions from each weight are summed to reconstruct the input. We compare SANs using the five previously defined activation functions on a variety of datasets (Physionet, UCI-epilepsy, MNIST, FMNIST) and show that models that are selected using $\varphi$ have small description representation length and consist of interpretable kernels.
Ensembling Strategies for Answering Natural Questions
Authors: Anthony Ferritto, Lin Pan, Rishav Chakravarti, Salim Roukos, Radu Florian, J. William Murdock, Avirup Sil
Date: 2019-10-30 00:00:00
Tasks: Natural Questions, Question Answering

Many of the top question answering systems today utilize ensembling to improve their performance on tasks such as the Stanford Question Answering Dataset (SQuAD) and Natural Questions (NQ) challenges. Unfortunately most of these systems do not publish their ensembling strategies used in their leaderboard submissions. In this work, we investigate a number of ensembling techniques and demonstrate a strategy which improves our F1 score for short answers on the dev set for NQ by 2.3 F1 points over our single model (which outperforms the previous SOTA by 1.9 F1 points).
Learning Without Loss
Authors: Veit Elser
Date: 2019-10-29 00:00:00
Tasks: Retrieval

We explore a new approach for training neural networks where all loss functions are replaced by hard constraints. The same approach is very successful in phase retrieval, where signals are reconstructed from magnitude constraints and general characteristics (sparsity, support, etc.). Instead of taking gradient steps, the optimizer in the constraint based approach, called relaxed-reflect-reflect (RRR), derives its steps from projections to local constraints. In neural networks one such projection makes the minimal modification to the inputs $x$, the associated weights $w$, and the pre-activation value $y$ at each neuron, to satisfy the equation $x\cdot w=y$. These projections, along with a host of other local projections (constraining pre- and post-activations, etc.) can be partitioned into two sets such that all the projections in each set can be applied concurrently, across the network and across all data in the training batch. This partitioning into two sets is analogous to the situation in phase retrieval and the setting for which the general purpose RRR optimizer was designed. Owing to the novelty of the method, this paper also serves as a self-contained tutorial. Starting with a single-layer network that performs non-negative matrix factorization, and concluding with a generative model comprising an autoencoder and classifier, all applications and their implementations by projections are described in complete detail. Although the new approach has the potential to extend the scope of neural networks (e.g. by defining activation not through functions but constraint sets), most of the featured models are standard to allow comparison with stochastic gradient descent.
Towards calibrated and scalable uncertainty representations for neural networks
Authors: Nabeel Seedat, Christopher Kanan
Date: 2019-10-28 00:00:00
Tasks: parameter estimation

For many applications it is critical to know the uncertainty of a neural network's predictions. While a variety of neural network parameter estimation methods have been proposed for uncertainty estimation, they have not been rigorously compared across uncertainty measures. We assess four of these parameter estimation methods to calibrate uncertainty estimation using four different uncertainty measures: entropy, mutual information, aleatoric uncertainty and epistemic uncertainty. We evaluate the calibration of these parameter estimation methods using expected calibration error. Additionally, we propose a novel method of neural network parameter estimation called RECAST, which combines cosine annealing with warm restarts with Stochastic Gradient Langevin Dynamics, capturing more diverse parameter distributions. When benchmarked against mutilated image data, we show that RECAST is well-calibrated and when combined with predictive entropy and epistemic uncertainty it offers the best calibrated measure of uncertainty when compared to recent methods.
Deep Learning of Subsurface Flow via Theory-guided Neural Network
Authors: Nanzhe Wang, Dongxiao Zhang, Haibin Chang, Heng Li
Date: 2019-10-24 00:00:00
Tasks: Deep Learning, Transfer Learning

Active researches are currently being performed to incorporate the wealth of scientific knowledge into data-driven approaches (e.g., neural networks) in order to improve the latter's effectiveness. In this study, the Theory-guided Neural Network (TgNN) is proposed for deep learning of subsurface flow. In the TgNN, as supervised learning, the neural network is trained with available observations or simulation data while being simultaneously guided by theory (e.g., governing equations, other physical constraints, engineering controls, and expert knowledge) of the underlying problem. The TgNN can achieve higher accuracy than the ordinary Artificial Neural Network (ANN) because the former provides physically feasible predictions and can be more readily generalized beyond the regimes covered with the training data. Furthermore, the TgNN model is proposed for subsurface flow with heterogeneous model parameters. Several numerical cases of two-dimensional transient saturated flow are introduced to test the performance of the TgNN. In the learning process, the loss function contains data mismatch, as well as PDE constraint, engineering control, and expert knowledge. After obtaining the parameters of the neural network by minimizing the loss function, a TgNN model is built that not only fits the data, but also adheres to physical/engineering constraints. Predicting the future response can be easily realized by the TgNN model. In addition, the TgNN model is tested in more complicated scenarios, such as prediction with changed boundary conditions, learning from noisy data or outliers, transfer learning, and engineering controls. Numerical results demonstrate that the TgNN model achieves much better predictability, reliability, and generalizability than ANN models due to the physical/engineering constraints in the former.
A categorisation and implementation of digital pen features for behaviour characterisation
Authors: Alexander Prange, Michael Barz, Daniel Sonntag
Date: 2018-10-01 00:00:00

In this paper we provide a categorisation and implementation of digital ink features for behaviour characterisation. Based on four feature sets taken from literature, we provide a categorisation in different classes of syntactic and semantic features. We implemented a publicly available framework to calculate these features and show its deployment in the use case of analysing cognitive assessments performed using a digital pen.
Poisson Multi-Bernoulli Approximations for Multiple Extended Object Filtering
Authors: Yuxuan Xia, Karl Granström, Lennart Svensson, Maryam Fatemi, Ángel F. García-Fernández, Jason L. Williams
Date: 2018-01-04 00:00:00
Tasks: Object

The Poisson multi-Bernoulli mixture (PMBM) is a multi-object conjugate prior for the closed-form Bayes random finite sets filter. The extended object PMBM filter provides a closed-form solution for multiple extended object filtering with standard models. This paper considers computationally lighter alternatives to the extended object PMBM filter by propagating a Poisson multi-Bernoulli (PMB) density through the filtering recursion. A new local hypothesis representation is presented where each measurement creates a new Bernoulli component. This facilitates the developments of methods for efficiently approximating the PMBM posterior density after the update step as a PMB. Based on the new hypothesis representation, two approximation methods are presented: one is based on the track-oriented multi-Bernoulli (MB) approximation, and the other is based on the variational MB approximation via Kullback-Leibler divergence minimisation. The performance of the proposed PMB filters with gamma Gaussian inverse-Wishart implementations are evaluated in a simulation study.
Objective Human Affective Vocal Expression Detection and Automatic Classification with Stochastic Models and Learning Systems
Authors: V. Vieira, R. Coelho, F. Assis
Date: 2019-10-04 00:00:00
Tasks: Classification, Emotion Classification

This paper presents a widespread analysis of affective vocal expression classification systems. In this study, state-of-the-art acoustic features are compared to two novel affective vocal prints for the detection of emotional states: the Hilbert-Huang-Hurst Coefficients (HHHC) and the vector of index of non-stationarity (INS). HHHC is here proposed as a nonlinear vocal source feature vector that represents the affective states according to their effects on the speech production mechanism. Emotional states are highlighted by the empirical mode decomposition (EMD) based method, which exploits the non-stationarity of the affective acoustic variations. Hurst coefficients (closely related to the excitation source) are then estimated from the decomposition process to compose the feature vector. Additionally, the INS vector is introduced as dynamic information to the HHHC feature. The proposed features are evaluated in speech emotion classification experiments with three databases in German and English languages. Three state-of-the-art acoustic features are adopted as baseline. The $\alpha$-integrated Gaussian model ($\alpha$-GMM) is also introduced for the emotion representation and classification. Its performance is compared to competing stochastic and machine learning classifiers. Results demonstrate that HHHC leads to significant classification improvement when compared to the baseline acoustic features. Moreover, results also show that $\alpha$-GMM outperforms the competing classification methods. Finally, HHHC and INS are also evaluated as complementary features for the GeMAPS and eGeMAPS feature sets