pwc-archive/papers-with-abstracts huggingface dataset:
Pay Attention when Required
Authors: Swetha Mandava, Szymon Migacz, Alex Fit Florea
Date: 2020-09-09 00:00:00
Tasks: Language Modelling, Paraphrase Identification, Question Answering, Sentiment Analysis

Transformer-based models consist of interleaved feed-forward blocks - that capture content meaning, and relatively more expensive self-attention blocks - that capture context meaning. In this paper, we explored trade-offs and ordering of the blocks to improve upon the current Transformer architecture and proposed PAR Transformer. It needs 35% lower compute time than Transformer-XL achieved by replacing ~63% of the self-attention blocks with feed-forward blocks, and retains the perplexity on WikiText-103 language modelling benchmark. We further validated our results on text8 and enwiki8 datasets, as well as on the BERT model.
A Survey on Data Pricing: from Economics to Data Science
Authors: Jian Pei
Date: 2020-09-09 00:00:00
Tasks: BIG-bench Machine Learning, Management, Marketing, Survey

Data are invaluable. How can we assess the value of data objectively, systematically and quantitatively? Pricing data, or information goods in general, has been studied and practiced in dispersed areas and principles, such as economics, marketing, electronic commerce, data management, data mining and machine learning. In this article, we present a unified, interdisciplinary and comprehensive overview of this important direction. We examine various motivations behind data pricing, understand the economics of data pricing and review the development and evolution of pricing models according to a series of fundamental principles. We discuss both digital products and data products. We also consider a series of challenges and directions for future work.
Discovering Textual Structures: Generative Grammar Induction using Template Trees
Authors: Thomas Winters, Luc De Raedt
Date: 2020-09-09 00:00:00
Tasks: Text Generation

Natural language generation provides designers with methods for automatically generating text, e.g. for creating summaries, chatbots and game content. In practise, text generators are often either learned and hard to interpret, or created by hand using techniques such as grammars and templates. In this paper, we introduce a novel grammar induction algorithm for learning interpretable grammars for generative purposes, called Gitta. We also introduce the novel notion of template trees to discover latent templates in corpora to derive these generative grammars. By using existing human-created grammars, we found that the algorithm can reasonably approximate these grammars using only a few examples. These results indicate that Gitta could be used to automatically learn interpretable and easily modifiable grammars, and thus provide a stepping stone for human-machine co-creation of generative models.
On Artificial Life and Emergent Computation in Physical Substrates
Authors: Kristine Heiney, Gunnar Tufte, Stefano Nichele
Date: 2020-09-09 00:00:00
Tasks: Artificial Life, Distributed Computing

In living systems, we often see the emergence of the ingredients necessary for computation -- the capacity for information transmission, storage, and modification -- begging the question of how we may exploit or imitate such biological systems in unconventional computing applications. What can we gain from artificial life in the advancement of computing technology? Artificial life provides us with powerful tools for understanding the dynamic behavior of biological systems and capturing this behavior in manmade substrates. With this approach, we can move towards a new computing paradigm concerned with harnessing emergent computation in physical substrates not governed by the constraints of Moore's law and ultimately realize massively parallel and distributed computing technology. In this paper, we argue that the lens of artificial life offers valuable perspectives for the advancement of high-performance computing technology. We first present a brief foundational background on artificial life and some relevant tools that may be applicable to unconventional computing. Two specific substrates are then discussed in detail: biological neurons and ensembles of nanomagnets. These substrates are the focus of the authors' ongoing work, and they are illustrative of the two sides of the approach outlined here -- the close study of living systems and the construction of artificial systems to produce life-like behaviors. We conclude with a philosophical discussion on what we can learn from approaching computation with the curiosity inherent to the study of artificial life. The main contribution of this paper is to present the great potential of using artificial life methodologies to uncover and harness the inherent computational power of physical substrates toward applications in unconventional high-performance computing.
Narrative Maps: An Algorithmic Approach to Represent and Extract Information Narratives
Authors: Brian Keith, Tanushree Mitra
Date: 2020-09-09 00:00:00
Tasks: Misinformation

Narratives are fundamental to our perception of the world and are pervasive in all activities that involve the representation of events in time. Yet, modern online information systems do not incorporate narratives in their representation of events occurring over time. This article aims to bridge this gap, combining the theory of narrative representations with the data from modern online systems. We make three key contributions: a theory-driven computational representation of narratives, a novel extraction algorithm to obtain these representations from data, and an evaluation of our approach. In particular, given the effectiveness of visual metaphors, we employ a route map metaphor to design a narrative map representation. The narrative map representation illustrates the events and stories in the narrative as a series of landmarks and routes on the map. Each element of our representation is backed by a corresponding element from formal narrative theory, thus providing a solid theoretical background to our method. Our approach extracts the underlying graph structure of the narrative map using a novel optimization technique focused on maximizing coherence while respecting structural and coverage constraints. We showcase the effectiveness of our approach by performing a user evaluation to assess the quality of the representation, metaphor, and visualization. Evaluation results indicate that the Narrative Map representation is a powerful method to communicate complex narratives to individuals. Our findings have implications for intelligence analysts, computational journalists, and misinformation researchers.
Segmentation-free Estimation of Aortic Diameters from MRI Using Deep Learning
Authors: Axel Aguerreberry, Ezequiel de la Rosa, Alain Lalande, Elmer Fernandez
Date: 2020-09-09 00:00:00
Tasks: Decision Making

Accurate and reproducible measurements of the aortic diameters are crucial for the diagnosis of cardiovascular diseases and for therapeutic decision making. Currently, these measurements are manually performed by healthcare professionals, being time consuming, highly variable, and suffering from lack of reproducibility. In this work we propose a supervised deep learning method for the direct estimation of aortic diameters. The approach is devised and tested over 100 magnetic resonance angiography scans without contrast agent. All data was expert-annotated at six aortic locations typically used in clinical practice. Our approach makes use of a 3D+2D convolutional neural network (CNN) that takes as input a 3D scan and outputs the aortic diameter at a given location. In a 5-fold cross-validation comparison against a fully 3D CNN and against a 3D multiresolution CNN, our approach was consistently superior in predicting the aortic diameters. Overall, the 3D+2D CNN achieved a mean absolute error between 2.2-2.4 mm depending on the considered aortic location. These errors are less than 1 mm higher than the inter-observer variability. Thus, suggesting that our method makes predictions almost reaching the expert's performance. We conclude that the work allows to further explore automatic algorithms for direct estimation of anatomical structures without the necessity of a segmentation step. It also opens possibilities for the automation of cardiovascular measurements in clinical settings.
Variational Preparation of the Sachdev-Ye-Kitaev Thermofield Double
Authors: Vincent Paul Su
Date: 2020-09-09 00:00:00

We provide an algorithm for preparing the thermofield double (TFD) state of the Sachdev-Ye-Kitaev model without the need for an auxiliary bath. Following previous work, the TFD can be cast as the approximate ground state of a Hamiltonian, $H_{\text{TFD}}$. Using variational quantum circuits, we propose and implement a gradient-based algorithm for learning parameters that find this ground state, an application of the variational quantum eigensolver. Concretely, we find quantum circuits that prepare the ground state of $H_{\text{TFD}}$ for the $q=4$ SYK model up to $N=12$.
Aspect Classification for Legal Depositions
Authors: Saurabh Chakravarty, Satvik Chekuri, Maanav Mehrotra, Edward A. Fox
Date: 2020-09-09 00:00:00
Tasks: Classification, General Classification, Information Retrieval, Question Answering, Retrieval

Attorneys and others have a strong interest in having a digital library with suitable services (e.g., summarizing, searching, and browsing) to help them work with large corpora of legal depositions. Their needs often involve understanding the semantics of such documents. That depends in part on the role of the deponent, e.g., plaintiff, defendant, law enforcement personnel, expert, etc. In the case of tort litigation associated with property and casualty insurance claims, such as relating to an injury, it is important to know not only about liability, but also about events, accidents, physical conditions, and treatments. We hypothesize that a legal deposition consists of various aspects that are discussed as part of the deponent testimony. Accordingly, we developed an ontology of aspects in a legal deposition for accident and injury cases. Using that, we have developed a classifier that can identify portions of text for each of the aspects of interest. Doing so was complicated by the peculiarities of this genre, e.g., that deposition transcripts generally consist of data in the form of question-answer (QA) pairs. Accordingly, our automated system starts with pre-processing, and then transforms the QA pairs into a canonical form made up of declarative sentences. Classifying the declarative sentences that are generated, according to the aspect, can then help with downstream tasks such as summarization, segmentation, question-answering, and information retrieval. Our methods have achieved a classification F1 score of 0.83. Having the aspects classified with a good accuracy will help in choosing QA pairs that can be used as candidate summary sentences, and to generate an informative summary for legal professionals or insurance claim agents. Our methodology could be extended to legal depositions of other kinds, and to aid services like searching.
Hardware Aware Training for Efficient Keyword Spotting on General Purpose and Specialized Hardware
Authors: Peter Blouw, Gurshaant Malik, Benjamin Morcos, Aaron R. Voelker, Chris Eliasmith
Date: 2020-09-09 00:00:00
Tasks: Keyword Spotting

Keyword spotting (KWS) provides a critical user interface for many mobile and edge applications, including phones, wearables, and cars. As KWS systems are typically 'always on', maximizing both accuracy and power efficiency are central to their utility. In this work we use hardware aware training (HAT) to build new KWS neural networks based on the Legendre Memory Unit (LMU) that achieve state-of-the-art (SotA) accuracy and low parameter counts. This allows the neural network to run efficiently on standard hardware (212$\mu$W). We also characterize the power requirements of custom designed accelerator hardware that achieves SotA power efficiency of 8.79$\mu$W, beating general purpose low power hardware (a microcontroller) by 24x and special purpose ASICs by 16x.
Forecasting financial markets with semantic network analysis in the COVID-19 crisis
Authors: A. Fronzetti Colladon, S. Grassi, F. Ravazzolo, F. Violante
Date: 2020-09-09 00:00:00
Tasks: Position, Time Series, Time Series Analysis

This paper uses a new textual data index for predicting stock market data. The index is applied to a large set of news to evaluate the importance of one or more general economic-related keywords appearing in the text. The index assesses the importance of the economic-related keywords, based on their frequency of use and semantic network position. We apply it to the Italian press and construct indices to predict Italian stock and bond market returns and volatilities in a recent sample period, including the COVID-19 crisis. The evidence shows that the index captures the different phases of financial time series well. Moreover, results indicate strong evidence of predictability for bond market data, both returns and volatilities, short and long maturities, and stock market volatility.
Meta-learning based Alternating Minimization Algorithm for Non-convex Optimization
Authors: Jingyuan Xia, Shengxi Li, Jun-Jie Huang, Imad Jaimoukha, Deniz Gunduz
Date: 2020-09-09 00:00:00
Tasks: Matrix Completion, Meta-Learning

In this paper, we propose a novel solution for non-convex problems of multiple variables, especially for those typically solved by an alternating minimization (AM) strategy that splits the original optimization problem into a set of sub-problems corresponding to each variable, and then iteratively optimize each sub-problem using a fixed updating rule. However, due to the intrinsic non-convexity of the original optimization problem, the optimization can usually be trapped into spurious local minimum even when each sub-problem can be optimally solved at each iteration. Meanwhile, learning-based approaches, such as deep unfolding algorithms, are highly limited by the lack of labelled data and restricted explainability. To tackle these issues, we propose a meta-learning based alternating minimization (MLAM) method, which aims to minimize a partial of the global losses over iterations instead of carrying minimization on each sub-problem, and it tends to learn an adaptive strategy to replace the handcrafted counterpart resulting in advance on superior performance. Meanwhile, the proposed MLAM still maintains the original algorithmic principle, which contributes to a better interpretability. We evaluate the proposed method on two representative problems, namely, bi-linear inverse problem: matrix completion, and non-linear problem: Gaussian mixture models. The experimental results validate that our proposed approach outperforms AM-based methods in standard settings, and is able to achieve effective optimization in challenging cases while other comparing methods would typically fail.
A dataset and classification model for Malay, Hindi, Tamil and Chinese music
Authors: Fajilatun Nahar, Kat Agres, Balamurali BT, Dorien Herremans
Date: 2020-09-09 00:00:00
Tasks: Classification, General Classification

In this paper we present a new dataset, with musical excepts from the three main ethnic groups in Singapore: Chinese, Malay and Indian (both Hindi and Tamil). We use this new dataset to train different classification models to distinguish the origin of the music in terms of these ethnic groups. The classification models were optimized by exploring the use of different musical features as the input. Both high level features, i.e., musically meaningful features, as well as low level features, i.e., spectrogram based features, were extracted from the audio files so as to optimize the performance of the different classification models.
Interpreting Deep Glucose Predictive Models for Diabetic People Using RETAIN
Authors: Maxime De Bois, Mounîm A. El Yacoubi, Mehdi Ammi
Date: 2020-09-08 00:00:00

Progress in the biomedical field through the use of deep learning is hindered by the lack of interpretability of the models. In this paper, we study the RETAIN architecture for the forecasting of future glucose values for diabetic people. Thanks to its two-level attention mechanism, the RETAIN model is interpretable while remaining as efficient as standard neural networks. We evaluate the model on a real-world type-2 diabetic population and we compare it to a random forest model and a LSTM-based recurrent neural network. Our results show that the RETAIN model outperforms the former and equals the latter on common accuracy metrics and clinical acceptability metrics, thereby proving its legitimacy in the context of glucose level forecasting. Furthermore, we propose tools to take advantage of the RETAIN interpretable nature. As informative for the patients as for the practitioners, it can enhance the understanding of the predictions made by the model and improve the design of future glucose predictive models.
SNoRe: Scalable Unsupervised Learning of Symbolic Node Representations
Authors: Sebastian Mežnar, Nada Lavrač, Blaž Škrlj
Date: 2020-09-08 00:00:00
Tasks: Node Classification, Structural Node Embedding

Learning from complex real-life networks is a lively research area, with recent advances in learning information-rich, low-dimensional network node representations. However, state-of-the-art methods are not necessarily interpretable and are therefore not fully applicable to sensitive settings in biomedical or user profiling tasks, where explicit bias detection is highly relevant. The proposed SNoRe (Symbolic Node Representations) algorithm is capable of learning symbolic, human-understandable representations of individual network nodes, based on the similarity of neighborhood hashes which serve as features. SNoRe's interpretable features are suitable for direct explanation of individual predictions, which we demonstrate by coupling it with the widely used instance explanation tool SHAP to obtain nomograms representing the relevance of individual features for a given classification. To our knowledge, this is one of the first such attempts in a structural node embedding setting. In the experimental evaluation on eleven real-life datasets, SNoRe proved to be competitive to strong baselines, such as variational graph autoencoders, node2vec and LINE. The vectorized implementation of SNoRe scales to large networks, making it suitable for contemporary network learning and analysis tasks.
Weak Form Theory-guided Neural Network (TgNN-wf) for Deep Learning of Subsurface Single and Two-phase Flow
Authors: Rui Xu, Dongxiao Zhang, Miao Rong, Nanzhe Wang
Date: 2020-09-08 00:00:00
Tasks: Form

Deep neural networks (DNNs) are widely used as surrogate models in geophysical applications; incorporating theoretical guidance into DNNs has improved the generalizability. However, most of such approaches define the loss function based on the strong form of conservation laws (via partial differential equations, PDEs), which is subject to deteriorated accuracy when the PDE has high order derivatives or the solution has strong discontinuities. Herein, we propose a weak form theory-guided neural network (TgNN-wf), which incorporates the weak form formulation of the PDE into the loss function combined with data constraint and initial and boundary conditions regularizations to tackle the aforementioned difficulties. In the weak form, high order derivatives in the PDE can be transferred to the test functions by performing integration-by-parts, which reduces computational error. We use domain decomposition with locally defined test functions, which captures local discontinuity effectively. Two numerical cases demonstrate the superiority of the proposed TgNN-wf over the strong form TgNN, including the hydraulic head prediction for unsteady-state 2D single-phase flow problems and the saturation profile prediction for 1D two-phase flow problems. Results show that TgNN-wf consistently has higher accuracy than TgNN, especially when strong discontinuity in the solution is present. TgNN-wf also trains faster than TgNN when the number of integration subdomains is not too large (<10,000). Moreover, TgNN-wf is more robust to noises. Thus, the proposed TgNN-wf paves the way for which a variety of deep learning problems in the small data regime can be solved more accurately and efficiently.
Mutual Information for Explainable Deep Learning of Multiscale Systems
Authors: Søren Taverniers, Eric J. Hall, Markos A. Katsoulakis, Daniel M. Tartakovsky
Date: 2020-09-07 00:00:00
Tasks: Deep Learning, Uncertainty Quantification

Timely completion of design cycles for complex systems ranging from consumer electronics to hypersonic vehicles relies on rapid simulation-based prototyping. The latter typically involves high-dimensional spaces of possibly correlated control variables (CVs) and quantities of interest (QoIs) with non-Gaussian and possibly multimodal distributions. We develop a model-agnostic, moment-independent global sensitivity analysis (GSA) that relies on differential mutual information to rank the effects of CVs on QoIs. The data requirements of this information-theoretic approach to GSA are met by replacing computationally intensive components of the physics-based model with a deep neural network surrogate. Subsequently, the GSA is used to explain the network predictions, and the surrogate is deployed to close design loops. Viewed as an uncertainty quantification method for interrogating the surrogate, this framework is compatible with a wide variety of black-box models. We demonstrate that the surrogate-driven mutual information GSA provides useful and distinguishable rankings on two applications of interest in energy storage. Consequently, our information-theoretic GSA provides an "outer loop" for accelerated product design by identifying the most and least sensitive input directions and performing subsequent optimization over appropriately reduced parameter subspaces.
Attention based Writer Independent Handwriting Verification
Authors: Mohammad Abuzar Shaikh, Tiehang Duan, Mihir Chauhan, Sargur Srihari
Date: 2020-09-07 00:00:00
Tasks: Handwriting Verification

The task of writer verification is to provide a likelihood score for whether the queried and known handwritten image samples belong to the same writer or not. Such a task calls for the neural network to make it's outcome interpretable, i.e. provide a view into the network's decision making process. We implement and integrate cross-attention and soft-attention mechanisms to capture the highly correlated and salient points in feature space of 2D inputs. The attention maps serve as an explanation premise for the network's output likelihood score. The attention mechanism also allows the network to focus more on relevant areas of the input, thus improving the classification performance. Our proposed approach achieves a precision of 86\% for detecting intra-writer cases in CEDAR cursive "AND" dataset. Furthermore, we generate meaningful explanations for the provided decision by extracting attention maps from multiple levels of the network.
On the Effectiveness of Minisum Approval Voting in an Open Strategy Setting: An Agent-Based Approach
Authors: Joop van de Heijning, Stephan Leitner, Alexandra Rausch
Date: 2020-09-07 00:00:00

This work researches the impact of including a wider range of participants in the strategy-making process on the performance of organizations which operate in either moderately or highly complex environments. Agent-based simulation demonstrates that the increased number of ideas generated from larger and diverse crowds and subsequent preference aggregation lead to rapid discovery of higher peaks in the organization's performance landscape. However, this is not the case when the expansion in the number of participants is small. The results confirm the most frequently mentioned benefit in the Open Strategy literature: the discovery of better performing strategies.