pwc-archive/papers-with-abstracts huggingface dataset:
Temporal EigenPAC for dyslexia diagnosis
Authors: Nicolás Gallego-Molina, Marco Formoso, Andrés Ortiz, Francisco J. Martínez-Murcia, Juan L. Luque
Date: 2021-04-13 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Functional Connectivity

Electroencephalography signals allow to explore the functional activity of the brain cortex in a non-invasive way. However, the analysis of these signals is not straightforward due to the presence of different artifacts and the very low signal-to-noise ratio. Cross-Frequency Coupling (CFC) methods provide a way to extract information from EEG, related to the synchronization among frequency bands. However, CFC methods are usually applied in a local way, computing the interaction between phase and amplitude at the same electrode. In this work we show a method to compute PAC features among electrodes to study the functional connectivity. Moreover, this has been applied jointly with Principal Component Analysis to explore patterns related to Dyslexia in 7-years-old children. The developed methodology reveals the temporal evolution of PAC-based connectivity. Directions of greatest variance computed by PCA are called eigenPACs here, since they resemble the classical \textit{eigenfaces} representation. The projection of PAC data onto the eigenPACs provide a set of features that has demonstrates their discriminative capability, specifically in the Beta-Gamma bands.
Hierarchical Adaptive Pooling by Capturing High-order Dependency for Graph Representation Learning
Authors: Ning Liu, Songlei Jian, Dongsheng Li, Yiming Zhang, Zhiquan Lai, Hongzuo Xu
Date: 2021-04-13 00:00:00
Tasks: Graph Classification, Graph Matching, Graph Representation Learning, Graph Similarity, Representation Learning

Graph neural networks (GNN) have been proven to be mature enough for handling graph-structured data on node-level graph representation learning tasks. However, the graph pooling technique for learning expressive graph-level representation is critical yet still challenging. Existing pooling methods either struggle to capture the local substructure or fail to effectively utilize high-order dependency, thus diminishing the expression capability. In this paper we propose HAP, a hierarchical graph-level representation learning framework, which is adaptively sensitive to graph structures, i.e., HAP clusters local substructures incorporating with high-order dependencies. HAP utilizes a novel cross-level attention mechanism MOA to naturally focus more on close neighborhood while effectively capture higher-order dependency that may contain crucial information. It also learns a global graph content GCont that extracts the graph pattern properties to make the pre- and post-coarsening graph content maintain stable, thus providing global guidance in graph coarsening. This novel innovation also facilitates generalization across graphs with the same form of features. Extensive experiments on fourteen datasets show that HAP significantly outperforms twelve popular graph pooling methods on graph classification task with an maximum accuracy improvement of 22.79%, and exceeds the performance of state-of-the-art graph matching and graph similarity learning algorithms by over 3.5% and 16.7%.
Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness
Authors: Max Revay, Ruigang Wang, Ian R. Manchester
Date: 2021-04-13 00:00:00
Tasks: All

This paper introduces recurrent equilibrium networks (RENs), a new class of nonlinear dynamical models} for applications in machine learning, system identification and control. The new model class admits ``built in'' behavioural guarantees of stability and robustness. All models in the proposed class are contracting -- a strong form of nonlinear stability -- and models can satisfy prescribed incremental integral quadratic constraints (IQC), including Lipschitz bounds and incremental passivity. RENs are otherwise very flexible: they can represent all stable linear systems, all previously-known sets of contracting recurrent neural networks and echo state networks, all deep feedforward neural networks, and all stable Wiener/Hammerstein models, and can approximate all fading-memory and contracting nonlinear systems. RENs are parameterized directly by a vector in R^N, i.e. stability and robustness are ensured without parameter constraints, which simplifies learning since \HL{generic methods for unconstrained optimization such as stochastic gradient descent and its variants can be used}. The performance and robustness of the new model set is evaluated on benchmark nonlinear system identification problems, and the paper also presents applications in data-driven nonlinear observer design and control with stability guarantees.
Multiple regression techniques for modeling dates of first performances of Shakespeare-era plays
Authors: Pablo Moscato, Hugh Craig, Gabriel Egan, Mohammad Nazmul Haque, Kevin Huang, Julia Sloan, Jon Corrales de Oliveira
Date: 2021-04-13 00:00:00
Tasks: Continued fraction, regression

The date of the first performance of a play of Shakespeare's time must usually be guessed with reference to multiple indirect external sources, or to some aspect of the content or style of the play. Identifying these dates is important to literary history and to accounts of developing authorial styles, such as Shakespeare's. In this study, we took a set of Shakespeare-era plays (181 plays from the period 1585--1610), added the best-guess dates for them from a standard reference work as metadata, and calculated a set of probabilities of individual words in these samples. We applied 11 regression methods to predict the dates of the plays at an 80/20 training/test split. We withdrew one play at a time, used the best-guess date metadata with the probabilities and weightings to infer its date, and thus built a model of date-probabilities interaction. We introduced a memetic algorithm-based Continued Fraction Regression (CFR) which delivered models using a small number of variables, leading to an interpretable model and reduced dimensionality. An in-depth analysis of the most commonly occurring 20 words in the CFR models in 100 independent runs helps explain the trends in linguistic and stylistic terms. The analysis with the subset of words revealed an interesting correlation of signature words with the Shakespeare-era play's genre.
On the validity of kernel approximations for orthogonally-initialized neural networks
Authors: James Martens
Date: 2021-04-13 00:00:00

In this note we extend kernel function approximation results for neural networks with Gaussian-distributed weights to single-layer networks initialized using Haar-distributed random orthogonal matrices (with possible rescaling). This is accomplished using recent results from random matrix theory.
Equivalence of quantum barren plateaus to cost concentration and narrow gorges
Authors: Andrew Arrasmith, Zoë Holmes, M. Cerezo, Patrick J. Coles
Date: 2021-04-12 00:00:00

Optimizing parameterized quantum circuits (PQCs) is the leading approach to make use of near-term quantum computers. However, very little is known about the cost function landscape for PQCs, which hinders progress towards quantum-aware optimizers. In this work, we investigate the connection between three different landscape features that have been observed for PQCs: (1) exponentially vanishing gradients (called barren plateaus), (2) exponential cost concentration about the mean, and (3) the exponential narrowness of minina (called narrow gorges). We analytically prove that these three phenomena occur together, i.e., when one occurs then so do the other two. A key implication of this result is that one can numerically diagnose barren plateaus via cost differences rather than via the computationally more expensive gradients. More broadly, our work shows that quantum mechanics rules out certain cost landscapes (which otherwise would be mathematically possible), and hence our results are interesting from a quantum foundations perspective.
On Representation Learning for Scientific News Articles Using Heterogeneous Knowledge Graphs
Authors: Angelika Romanou, Panayiotis Smeros, Karl Aberer
Date: 2021-04-12 00:00:00
Tasks: Articles, Fact Checking, Graph Neural Network, Knowledge Graphs, Knowledge Tracing, Link Prediction, Misinformation, Representation Learning

In the era of misinformation and information inflation, the credibility assessment of the produced news is of the essence. However, fact-checking can be challenging considering the limited references presented in the news. This challenge can be transcended by utilizing the knowledge graph that is related to the news articles. In this work, we present a methodology for creating scientific news article representations by modeling the directed graph between the scientific news articles and the cited scientific publications. The network used for the experiments is comprised of the scientific news articles, their topic, the cited research literature, and their corresponding authors. We implement and present three different approaches: 1) a baseline Relational Graph Convolutional Network (R-GCN), 2) a Heterogeneous Graph Neural Network (HetGNN) and 3) a Heterogeneous Graph Transformer (HGT). We test these models in the downstream task of link prediction on the: a) news article - paper links and b) news article - article topic links. The results show promising applications of graph neural network approaches in the domains of knowledge tracing and scientific news credibility assessment.
Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviews
Authors: Mohammad Abdul Hadi, Fatemeh H. Fard
Date: 2021-04-12 00:00:00
Tasks: Classification, General Classification, Multi-class Classification, zero-shot-classification, Zero-Shot Learning

Context: Mobile app reviews written by users on app stores or social media are significant resources for app developers.Analyzing app reviews have proved to be useful for many areas of software engineering (e.g., requirement engineering, testing). Automatic classification of app reviews requires extensive efforts to manually curate a labeled dataset. When the classification purpose changes (e.g. identifying bugs versus usability issues or sentiment), new datasets should be labeled, which prevents the extensibility of the developed models for new desired classes/tasks in practice. Recent pre-trained neural language models (PTM) are trained on large corpora in an unsupervised manner and have found success in solving similar Natural Language Processing problems. However, the applicability of PTMs is not explored for app review classification Objective: We investigate the benefits of PTMs for app review classification compared to the existing models, as well as the transferability of PTMs in multiple settings. Method: We empirically study the accuracy and time efficiency of PTMs compared to prior approaches using six datasets from literature. In addition, we investigate the performance of the PTMs trained on app reviews (i.e. domain-specific PTMs) . We set up different studies to evaluate PTMs in multiple settings: binary vs. multi-class classification, zero-shot classification (when new labels are introduced to the model), multi-task setting, and classification of reviews from different resources. The datasets are manually labeled app review datasets from Google Play Store, Apple App Store, and Twitter data. In all cases, Micro and Macro Precision, Recall, and F1-scores will be used and we will report the time required for training and prediction with the models.
Contextual HyperNetworks for Novel Feature Adaptation
Authors: Angus Lamb, Evgeny Saveliev, Yingzhen Li, Sebastian Tschiatschek, Camilla Longden, Simon Woodhead, José Miguel Hernández-Lobato, Richard E. Turner, Pashmina Cameron, Cheng Zhang
Date: 2021-04-12 00:00:00
Tasks: Few-Shot Learning, Imputation, Meta-Learning, Recommendation Systems

While deep learning has obtained state-of-the-art results in many applications, the adaptation of neural network architectures to incorporate new output features remains a challenge, as neural networks are commonly trained to produce a fixed output dimension. This issue is particularly severe in online learning settings, where new output features, such as items in a recommender system, are added continually with few or no associated observations. As such, methods for adapting neural networks to novel features which are both time and data-efficient are desired. To address this, we propose the Contextual HyperNetwork (CHN), an auxiliary model which generates parameters for extending the base model to a new feature, by utilizing both existing data as well as any observations and/or metadata associated with the new feature. At prediction time, the CHN requires only a single forward pass through a neural network, yielding a significant speed-up when compared to re-training and fine-tuning approaches. To assess the performance of CHNs, we use a CHN to augment a partial variational autoencoder (P-VAE), a deep generative model which can impute the values of missing features in sparsely-observed data. We show that this system obtains improved few-shot learning performance for novel features over existing imputation and meta-learning baselines across recommender systems, e-learning, and healthcare tasks.
Enhancing User' s Income Estimation with Super-App Alternative Data
Authors: Gabriel Suarez, Juan Raful, Maria A. Luque, Carlos F. Valencia, Alejandro Correa-Bahnsen
Date: 2021-04-12 00:00:00

This paper presents the advantages of alternative data from Super-Apps to enhance user' s income estimation models. It compares the performance of these alternative data sources with the performance of industry-accepted bureau income estimators that takes into account only financial system information; successfully showing that the alternative data manage to capture information that bureau income estimators do not. By implementing the TreeSHAP method for Stochastic Gradient Boosting Interpretation, this paper highlights which of the customer' s behavioral and transactional patterns within a Super-App have a stronger predictive power when estimating user' s income. Ultimately, this paper shows the incentive for financial institutions to seek to incorporate alternative data into constructing their risk profiles.
Evidence-based Prescriptive Analytics, CAUSAL Digital Twin and a Learning Estimation Algorithm
Authors: PG Madhavan
Date: 2021-04-12 00:00:00
Tasks: counterfactual

Evidence-based Prescriptive Analytics (EbPA) is necessary to determine optimal operational set-points that will improve business productivity. EbPA results from what-if analysis and counterfactual experimentation on CAUSAL Digital Twins (CDTs) that quantify cause-effect relationships in the DYNAMICS of a system of connected assets. We describe the basics of Causality and Causal Graphs and develop a Learning Causal Digital Twin (LCDT) solution; our algorithm uses a simple recurrent neural network with some innovative modifications incorporating Causal Graph simulation. Since LCDT is a learning digital twin where parameters are learned online in real-time with minimal pre-configuration, the work of deploying digital twins will be significantly simplified. A proof-of-principle of LCDT was conducted using real vibration data from a system of bearings; results of causal factor estimation, what-if analysis study and counterfactual experiment are very encouraging.
Efficient Optimal Transport Algorithm by Accelerated Gradient descent
Authors: Dongsheng An, Na lei, Xianfeng GU
Date: 2021-04-12 00:00:00

Optimal transport (OT) plays an essential role in various areas like machine learning and deep learning. However, computing discrete optimal transport plan for large scale problems with adequate accuracy and efficiency is still highly challenging. Recently, methods based on the Sinkhorn algorithm add an entropy regularizer to the prime problem and get a trade off between efficiency and accuracy. In this paper, we propose a novel algorithm to further improve the efficiency and accuracy based on Nesterov's smoothing technique. Basically, the non-smooth c-transform of the Kantorovich potential is approximated by the smooth Log-Sum-Exp function, which finally smooths the original non-smooth Kantorovich dual functional (energy). The smooth Kantorovich functional can be optimized by the fast proximal gradient algorithm (FISTA) efficiently. Theoretically, the computational complexity of the proposed method is given by $O(n^{\frac{5}{2}} \sqrt{\log n} /\epsilon)$, which is lower than that of the Sinkhorn algorithm. Empirically, compared with the Sinkhorn algorithm, our experimental results demonstrate that the proposed method achieves faster convergence and better accuracy with the same parameter.
Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generation
Authors: Sarik Ghazarian, Zixi Liu, Akash SM, Ralph Weischedel, Aram Galstyan, Nanyun Peng
Date: 2021-04-12 00:00:00
Proceeding: NAACL 2021 4
Tasks: Story Generation

With the recent advances of open-domain story generation, the lack of reliable automatic evaluation metrics becomes an increasingly imperative issue that hinders the fast development of story generation. According to conducted researches in this regard, learnable evaluation metrics have promised more accurate assessments by having higher correlations with human judgments. A critical bottleneck of obtaining a reliable learnable evaluation metric is the lack of high-quality training data for classifiers to efficiently distinguish plausible and implausible machine-generated stories. Previous works relied on \textit{heuristically manipulated} plausible examples to mimic possible system drawbacks such as repetition, contradiction, or irrelevant content in the text level, which can be \textit{unnatural} and \textit{oversimplify} the characteristics of implausible machine-generated stories. We propose to tackle these issues by generating a more comprehensive set of implausible stories using {\em plots}, which are structured representations of controllable factors used to generate stories. Since these plots are compact and structured, it is easier to manipulate them to generate text with targeted undesirable properties, while at the same time maintain the grammatical correctness and naturalness of the generated sentences. To improve the quality of generated implausible stories, we further apply the adversarial filtering procedure presented by \citet{zellers2018swag} to select a more nuanced set of implausible texts. Experiments show that the evaluation metrics trained on our generated data result in more reliable automatic assessments that correlate remarkably better with human judgments compared to the baselines.
Practical Defences Against Model Inversion Attacks for Split Neural Networks
Authors: Tom Titcombe, Adam J. Hall, Pavlos Papadopoulos, Daniele Romanini
Date: 2021-04-12 00:00:00
Tasks: Federated Learning

We describe a threat model under which a split network-based federated learning system is susceptible to a model inversion attack by a malicious computational server. We demonstrate that the attack can be successfully performed with limited knowledge of the data distribution by the attacker. We propose a simple additive noise method to defend against model inversion, finding that the method can significantly reduce attack efficacy at an acceptable accuracy trade-off on MNIST. Furthermore, we show that NoPeekNN, an existing defensive method, protects different information from exposure, suggesting that a combined defence is necessary to fully protect private user data.
Active learning for medical code assignment
Authors: Martha Dais Ferreira, Michal Malyska, Nicola Sahar, Riccardo Miotto, Fernando Paulovich, Evangelos Milios
Date: 2021-04-12 00:00:00
Tasks: Active Learning, Decision Making, General Classification, Multi Label Text Classification, Multi-Label Text Classification, text-classification, Text Classification

Machine Learning (ML) is widely used to automatically extract meaningful information from Electronic Health Records (EHR) to support operational, clinical, and financial decision-making. However, ML models require a large number of annotated examples to provide satisfactory results, which is not possible in most healthcare scenarios due to the high cost of clinician-labeled data. Active Learning (AL) is a process of selecting the most informative instances to be labeled by an expert to further train a supervised algorithm. We demonstrate the effectiveness of AL in multi-label text classification in the clinical domain. In this context, we apply a set of well-known AL methods to help automatically assign ICD-9 codes on the MIMIC-III dataset. Our results show that the selection of informative instances provides satisfactory classification with a significantly reduced training set (8.3\% of the total instances). We conclude that AL methods can significantly reduce the manual annotation cost while preserving model performance.
Predicting the Accuracy of Early-est Earthquake Magnitude Estimates with an LSTM Neural Network: A Preliminary Analysis
Authors: Massimo Nazaria
Date: 2021-04-12 00:00:00

This report presents a preliminary analysis of an LSTM neural network designed to predict the accuracy of magnitude estimates computed by Early-est during the first minutes after an earthquake occurs.
Real-time Forecast Models for TBM Load Parameters Based on Machine Learning Methods
Authors: Xianjie Gao, Xueguan Song, Maolin Shi, Chao Zhang, Hongwei Zhang
Date: 2021-04-12 00:00:00
Tasks: BIG-bench Machine Learning

Because of the fast advance rate and the improved personnel safety, tunnel boring machines (TBMs) have been widely used in a variety of tunnel construction projects. The dynamic modeling of TBM load parameters (including torque, advance rate and thrust) plays an essential part in the design, safe operation and fault prognostics of this complex engineering system. In this paper, based on in-situ TBM operational data, we use the machine-learning (ML) methods to build the real-time forecast models for TBM load parameters, which can instantaneously provide the future values of the TBM load parameters as long as the current data are collected. To decrease the model complexity and improve the generalization, we also apply the least absolute shrinkage and selection (Lasso) method to extract the essential features of the forecast task. The experimental results show that the forecast models based on deep-learning methods, {\it e.g.}, recurrent neural network and its variants, outperform the ones based on the shallow-learning methods, {\it e.g.}, support vector regression and random forest. Moreover, the Lasso-based feature extraction significantly improves the performance of the resultant models.
Mediators in Determining what Processing BERT Performs First
Authors: Aviv Slobodkin, Leshem Choshen, Omri Abend
Date: 2021-04-13 00:00:00
Proceeding: NAACL 2021 4

Probing neural models for the ability to perform downstream tasks using their activation patterns is often used to localize what parts of the network specialize in performing what tasks. However, little work addressed potential mediating factors in such comparisons. As a test-case mediating factor, we consider the prediction's context length, namely the length of the span whose processing is minimally required to perform the prediction. We show that not controlling for context length may lead to contradictory conclusions as to the localization patterns of the network, depending on the distribution of the probing dataset. Indeed, when probing BERT with seven tasks, we find that it is possible to get 196 different rankings between them when manipulating the distribution of context lengths in the probing dataset. We conclude by presenting best practices for conducting such comparisons in the future.