pwc-archive/papers-with-abstracts huggingface dataset:
Nonlinear Level Set Learning for Function Approximation on Sparse Data with Applications to Parametric Differential Equations
Authors: Anthony Gruber, Max Gunzburger, Lili Ju, Yuankai Teng, Zhu Wang
Date: 2021-04-29 00:00:00
Tasks: Dimensionality Reduction

A dimension reduction method based on the "Nonlinear Level set Learning" (NLL) approach is presented for the pointwise prediction of functions which have been sparsely sampled. Leveraging geometric information provided by the Implicit Function Theorem, the proposed algorithm effectively reduces the input dimension to the theoretical lower bound with minor accuracy loss, providing a one-dimensional representation of the function which can be used for regression and sensitivity analysis. Experiments and applications are presented which compare this modified NLL with the original NLL and the Active Subspaces (AS) method. While accommodating sparse input data, the proposed algorithm is shown to train quickly and provide a much more accurate and informative reduction than either AS or the original NLL on two example functions with high-dimensional domains, as well as two state-dependent quantities depending on the solutions to parametric differential equations.
Continuous Decoding of Daily-Life Hand Movements from Forearm Muscle Activity for Enhanced Myoelectric Control of Hand Prostheses
Authors: Alessandro Salatiello, Martin A. Giese
Date: 2021-04-29 00:00:00

State-of-the-art motorized hand prostheses are endowed with actuators able to provide independent and proportional control of as many as six degrees of freedom (DOFs). The control signals are derived from residual electromyographic (EMG) activity, recorded concurrently from relevant forearm muscles. Nevertheless, the functional mapping between forearm EMG activity and hand kinematics is only known with limited accuracy. Therefore, no robust method exists for the reliable computation of control signals for the independent and proportional actuation of more than two DOFs. A common approach to deal with this limitation is to pre-program the prostheses for the execution of a restricted number of behaviors (e.g., pinching, grasping, and wrist rotation) that are activated by the detection of specific EMG activation patterns. However, this approach severely limits the range of activities users can perform with the prostheses during their daily living. In this work, we introduce a novel method, based on a long short-term memory (LSTM) network, to continuously map forearm EMG activity onto hand kinematics. Critically, unlike previous work, which often focuses on simple and highly controlled motor tasks, we tested our method on a dataset of activities of daily living (ADLs): the KIN-MUS UJI dataset. To the best of our knowledge, ours is the first reported work on the prediction of hand kinematics that uses this challenging dataset. Remarkably, we show that our network is able to generalize to novel untrained ADLs. Our results suggest that the presented method is suitable for the generation of control signals for the independent and proportional actuation of the multiple DOFs of state-of-the-art hand prostheses.
Deep Neural Network as an alternative to Boosted Decision Trees for PID
Authors: Denis Stanev, Riccardo Riva, Michele Umassi
Date: 2021-04-28 00:00:00
Tasks: Binary Classification

In this paper we recreate, and improve, the binary classification method for particles proposed in Roe et al. (2005) paper "Boosted decision trees as an alternative to artificial neural networks for particle identification". Such particles are tau neutrinos, which we will refer to as background, and electronic neutrinos: the signal we are interested in. In the original paper the preferred algorithm is a Boosted decision tree. This is due to its low effort tuning and good overall performance at the time. Our choice for implementation is a deep neural network, faster and more promising in performance. We will show how, using modern techniques, we are able to improve on the original result, both in accuracy and in training time.
Dynamic Mode Decomposition in Adaptive Mesh Refinement and Coarsening Simulations
Authors: Gabriel F. Barros, Malú Grave, Alex Viguerie, Alessandro Reali, Alvaro L. G. A. Coutinho
Date: 2021-04-28 00:00:00

Dynamic Mode Decomposition (DMD) is a powerful data-driven method used to extract spatio-temporal coherent structures that dictate a given dynamical system. The method consists of stacking collected temporal snapshots into a matrix and mapping the nonlinear dynamics using a linear operator. The standard procedure considers that snapshots possess the same dimensionality for all the observable data. However, this often does not occur in numerical simulations with adaptive mesh refinement/coarsening schemes (AMR/C). This paper proposes a strategy to enable DMD to extract features from observations with different mesh topologies and dimensions, such as those found in AMR/C simulations. For this purpose, the adaptive snapshots are projected onto the same reference function space, enabling the use of snapshot-based methods such as DMD. The present strategy is applied to challenging AMR/C simulations: a continuous diffusion-reaction epidemiological model for COVID-19, a density-driven gravity current simulation, and a bubble rising problem. We also evaluate the DMD efficiency to reconstruct the dynamics and some relevant quantities of interest. In particular, for the SEIRD model and the bubble rising problem, we evaluate DMD's ability to extrapolate in time (short-time future estimates).
A Study of the Mathematics of Deep Learning
Authors: Anirbit Mukherjee
Date: 2021-04-28 00:00:00
Tasks: Deep Learning

"Deep Learning"/"Deep Neural Nets" is a technological marvel that is now increasingly deployed at the cutting-edge of artificial intelligence tasks. This dramatic success of deep learning in the last few years has been hinged on an enormous amount of heuristics and it has turned out to be a serious mathematical challenge to be able to rigorously explain them. In this thesis, submitted to the Department of Applied Mathematics and Statistics, Johns Hopkins University we take several steps towards building strong theoretical foundations for these new paradigms of deep-learning. In chapter 2 we show new circuit complexity theorems for deep neural functions and prove classification theorems about these function spaces which in turn lead to exact algorithms for empirical risk minimization for depth 2 ReLU nets. We also motivate a measure of complexity of neural functions to constructively establish the existence of high-complexity neural functions. In chapter 3 we give the first algorithm which can train a ReLU gate in the realizable setting in linear time in an almost distribution free set up. In chapter 4 we give rigorous proofs towards explaining the phenomenon of autoencoders being able to do sparse-coding. In chapter 5 we give the first-of-its-kind proofs of convergence for stochastic and deterministic versions of the widely used adaptive gradient deep-learning algorithms, RMSProp and ADAM. This chapter also includes a detailed empirical study on autoencoders of the hyper-parameter values at which modern algorithms have a significant advantage over classical acceleration based methods. In the last chapter 6 we give new and improved PAC-Bayesian bounds for the risk of stochastic neural nets. This chapter also includes an experimental investigation revealing new geometric properties of the paths in weight space that are traced out by the net during the training.
Analysis of Legal Documents via Non-negative Matrix Factorization Methods
Authors: Ryan Budahazy, Lu Cheng, Yihuan Huang, Andrew Johnson, Pengyu Li, Joshua Vendrow, Zhoutong Wu, Denali Molitor, Elizaveta Rebrova, Deanna Needell
Date: 2021-04-28 00:00:00

The California Innocence Project (CIP), a clinical law school program aiming to free wrongfully convicted prisoners, evaluates thousands of mails containing new requests for assistance and corresponding case files. Processing and interpreting this large amount of information presents a significant challenge for CIP officials, which can be successfully aided by topic modeling techniques.In this paper, we apply Non-negative Matrix Factorization (NMF) method and implement various offshoots of it to the important and previously unstudied data set compiled by CIP. We identify underlying topics of existing case files and classify request files by crime type and case status (decision type). The results uncover the semantic structure of current case files and can provide CIP officials with a general understanding of newly received case files before further examinations. We also provide an exposition of popular variants of NMF with their experimental results and discuss the benefits and drawbacks of each variant through the real-world application.
Algorithmic Factors Influencing Bias in Machine Learning
Authors: William Blanzeisky, Pádraig Cunningham
Date: 2021-04-28 00:00:00
Tasks: BIG-bench Machine Learning, Management

It is fair to say that many of the prominent examples of bias in Machine Learning (ML) arise from bias that is there in the training data. In fact, some would argue that supervised ML algorithms cannot be biased, they reflect the data on which they are trained. In this paper we demonstrate how ML algorithms can misrepresent the training data through underestimation. We show how irreducible error, regularization and feature and class imbalance can contribute to this underestimation. The paper concludes with a demonstration of how the careful management of synthetic counterfactuals can ameliorate the impact of this underestimation bias.
Simplified Kalman filter for online rating: one-fits-all approach
Authors: Leszek Szczecinski, Raphaëlle Tihon
Date: 2021-04-28 00:00:00
Tasks: All

In this work, we deal with the problem of rating in sports, where the skills of the players/teams are inferred from the observed outcomes of the games. Our focus is on the online rating algorithms which estimate the skills after each new game by exploiting the probabilistic models of the relationship between the skills and the game outcome. We propose a Bayesian approach which may be seen as an approximate Kalman filter and which is generic in the sense that it can be used with any skills-outcome model and can be applied in the individual -- as well as in the group-sports. We show how the well-know algorithms (such as the Elo, the Glicko, and the TrueSkill algorithms) may be seen as instances of the one-fits-all approach we propose. In order to clarify the conditions under which the gains of the Bayesian approach over the simpler solutions can actually materialize, we critically compare the known and the new algorithms by means of numerical examples using the synthetic as well as the empirical data.
Modelling Cooperation and Competition in Urban Retail Ecosystems with Complex Network Metrics
Authors: Jordan Cambe, Krittika D'Silva, Anastasios Noulas, Cecilia Mascolo, Adam Waksman
Date: 2021-04-28 00:00:00

Understanding the impact that a new business has on the local market ecosystem is a challenging task as it is multifaceted in nature. Past work in this space has examined the collaborative or competitive role of homogeneous venue types (i.e. the impact of a new bookstore on existing bookstores). However, these prior works have been limited in their scope and explanatory power. To better measure retail performance in a modern city, a model should consider a number of factors that interact synchronously. This paper is the first which considers the multifaceted types of interactions that occur in urban cities when examining the impact of new businesses. We first present a modeling framework which examines the role of new businesses in their respective local areas. Using a longitudinal dataset from location technology platform Foursquare, we model new venue impact across 26 major cities worldwide. Representing cities as connected networks of venues, we quantify their structure and characterise their dynamics over time. We note a strong community structure emerging in these retail networks, an observation that highlights the interplay of cooperative and competitive forces that emerge in local ecosystems of retail establishments. We next devise a data-driven metric that captures the first-order correlation on the impact of a new venue on retailers within its vicinity accounting for both homogeneous and heterogeneous interactions between venue types. Lastly, we build a supervised machine learning model to predict the impact of a given new venue on its local retail ecosystem. Our approach highlights the power of complex network measures in building machine learning prediction models. These models have numerous applications within the retail sector and can support policymakers, business owners, and urban planners in the development of models to characterize and predict changes in urban settings.
SMLSOM: The shrinking maximum likelihood self-organizing map
Authors: Ryosuke Motegi, Yoichi Seki
Date: 2021-04-28 00:00:00
Tasks: Clustering, Model Selection

Determining the number of clusters in a dataset is a fundamental issue in data clustering. Many methods have been proposed to solve the problem of selecting the number of clusters, considering it to be a problem with regard to model selection. This paper proposes an efficient algorithm that automatically selects a suitable number of clusters based on a probability distribution model framework. The algorithm includes the following two components. First, a generalization of Kohonen's self-organizing map (SOM) is introduced. In Kohonen's SOM, clusters are modeled as mean vectors. In the generalized SOM, each cluster is modeled as a probabilistic distribution and constructed by samples classified based on the likelihood. Second, the dynamically updating method of the SOM structure is introduced. In Kohonen's SOM, each cluster is tied to a node of a fixed two-dimensional lattice space and learned using neighborhood relations between nodes based on Euclidean distance. The extended SOM defines a graph with clusters as vertices and neighborhood relations as links and updates the graph structure by cutting weakly-connection and unnecessary vertex deletions. The weakness of a link is measured using the Kullback--Leibler divergence, and the redundancy of a vertex is measured using the minimum description length. Those extensions make it efficient to determine the appropriate number of clusters. Compared with existing methods, the proposed method is computationally efficient and can accurately select the number of clusters.
Personalized Keyphrase Detection using Speaker and Environment Information
Authors: Rajeev Rikhye, Quan Wang, Qiao Liang, Yanzhang He, Ding Zhao, Yiteng, Huang, Arun Narayanan, Ian McGraw
Date: 2021-04-28 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Speaker Separation, Speaker Verification, speech-recognition, Speech Recognition, Text-Independent Speaker Verification

In this paper, we introduce a streaming keyphrase detection system that can be easily customized to accurately detect any phrase composed of words from a large vocabulary. The system is implemented with an end-to-end trained automatic speech recognition (ASR) model and a text-independent speaker verification model. To address the challenge of detecting these keyphrases under various noisy conditions, a speaker separation model is added to the feature frontend of the speaker verification model, and an adaptive noise cancellation (ANC) algorithm is included to exploit cross-microphone noise coherence. Our experiments show that the text-independent speaker verification model largely reduces the false triggering rate of the keyphrase detection, while the speaker separation model and adaptive noise cancellation largely reduce false rejections.
Defined the predictors of the lightning over India by using artificial neural network
Authors: Pradip Kumar Gautam, Deweshvar Singh
Date: 2021-04-28 00:00:00
Tasks: regression, Time Series, Time Series Analysis

Lightning casualties cause tremendous loss to life and property. However, very lately lightning has been considered as one of the major natural calamities which is now studied or monitored with proper instrumentation. The lightning characteristics over India have been studying by using daily data low resolution time series and monthly data high resolution monthly climatology. We have used ANN time series method (a neural network) to analyze the time series and defined which one will be the best predictor of lightning over India. The time series of lightning is output(dependent) and input (independent) are k-index, AOD, Cape etc. The Gaussian process regression, support vector machine, regression trees and linear regression defined the input variables. Which show approximately linear relation.
Optimizing Rescoring Rules with Interpretable Representations of Long-Term Information
Authors: Aaron Fisher
Date: 2021-04-28 00:00:00
Tasks: Binary Classification

Analyzing temporal data (e.g., wearable device data) requires a decision about how to combine information from the recent and distant past. In the context of classifying sleep status from actigraphy, Webster's rescoring rules offer one popular solution based on the long-term patterns in the output of a moving-window model. Unfortunately, the question of how to optimize rescoring rules for any given setting has remained unsolved. To address this problem and expand the possible use cases of rescoring rules, we propose rephrasing these rules in terms of epoch-specific features. Our features take two general forms: (1) the time lag between now and the most recent [or closest upcoming] bout of time spent in a given state, and (2) the length of the most recent [or closest upcoming] bout of time spent in a given state. Given any initial moving window model, these features can be defined recursively, allowing for straightforward optimization of rescoring rules. Joint optimization of the moving window model and the subsequent rescoring rules can also be implemented using gradient-based optimization software, such as Tensorflow. Beyond binary classification problems (e.g., sleep-wake), the same approach can be applied to summarize long-term patterns for multi-state classification problems (e.g., sitting, walking, or stair climbing). We find that optimized rescoring rules improve the performance of sleep-wake classifiers, achieving accuracy comparable to that of certain neural network architectures.
Dynamical prediction of two meteorological factors using the deep neural network and the long short-term memory $(2)$
Authors: Ki-Hong Shin, Jae-Won Jung, Ki-Ho Chang, Dong-In Lee, Cheol-Hwan You, Kyungsik Kim
Date: 2021-04-28 00:00:00
Tasks: Time Series Analysis

This paper presents the predictive accuracy using two-variate meteorological factors, average temperature and average humidity, in neural network algorithms. We analyze result in five learning architectures such as the traditional artificial neural network, deep neural network, and extreme learning machine, long short-term memory, and long-short-term memory with peephole connections, after manipulating the computer-simulation. Our neural network modes are trained on the daily time-series dataset during seven years (from 2014 to 2020). From the trained results for 2500, 5000, and 7500 epochs, we obtain the predicted accuracies of the meteorological factors produced from outputs in ten metropolitan cities (Seoul, Daejeon, Daegu, Busan, Incheon, Gwangju, Pohang, Mokpo, Tongyeong, and Jeonju). The error statistics is found from the result of outputs, and we compare these values to each other after the manipulation of five neural networks. As using the long-short-term memory model in testing 1 (the average temperature predicted from the input layer with six input nodes), Tonyeong has the lowest root mean squared error (RMSE) value of 0.866 $(%)$ in summer from the computer-simulation in order to predict the temperature. To predict the humidity, the RMSE is shown the lowest value of 5.732 $(%)$, when using the long short-term memory model in summer in Mokpo in testing 2 (the average humidity predicted from the input layer with six input nodes). Particularly, the long short-term memory model is is found to be more accurate in forecasting daily levels than other neural network models in temperature and humidity forecastings. Our result may provide a computer-simuation basis for the necessity of exploring and develping a novel neural network evaluation method in the future.
Multi-class Text Classification using BERT-based Active Learning
Authors: Sumanth Prabhu, Moosa Mohamed, Hemant Misra
Date: 2021-04-27 00:00:00
Tasks: Active Learning, Classification, General Classification, Multi Class Text Classification, Natural Language Understanding, text-classification, Text Classification

Text Classification finds interesting applications in the pickup and delivery services industry where customers require one or more items to be picked up from a location and delivered to a certain destination. Classifying these customer transactions into multiple categories helps understand the market needs for different customer segments. Each transaction is accompanied by a text description provided by the customer to describe the products being picked up and delivered which can be used to classify the transaction. BERT-based models have proven to perform well in Natural Language Understanding. However, the product descriptions provided by the customers tend to be short, incoherent and code-mixed (Hindi-English) text which demands fine-tuning of such models with manually labelled data to achieve high accuracy. Collecting this labelled data can prove to be expensive. In this paper, we explore Active Learning strategies to label transaction descriptions cost effectively while using BERT to train a transaction classification model. On TREC-6, AG's News Corpus and an internal dataset, we benchmark the performance of BERT across different Active Learning strategies in Multi-Class Text Classification.
Identifying Hubs in Undergraduate Course Networks Based on Scaled Co-Enrollments: Extended Version
Authors: Gary M. Weiss, Nam Nguyen, Karla Dominguez, Daniel D. Leeds
Date: 2021-04-27 00:00:00

Understanding course enrollment patterns is valuable to predict upcoming demands for future courses, and to provide student with realistic courses to pursue given their current backgrounds. This study uses undergraduate student enrollment data to form networks of courses where connections are based on student co-enrollments. The course networks generated in this paper are based on eight years of undergraduate course enrollment data from a large metropolitan university. The networks are analyzed to identify "hub" courses often taken with many other courses. Two notions of hubs are considered: one focused on raw popularity across all students, and one focused on proportional likelihoods of co-enrollment with other courses. A variety of network metrics are calculated to evaluate the course networks. Academic departments and high-level academic categories, such as Humanities vs STEM, are studied for their influence over course groupings. The identification of hub courses has practical applications, since it can help better predict the impact of changes in course offerings and in course popularity, and in the case of interdisciplinary hub courses, can be used to increase or decrease interest and enrollments in specific academic departments and areas.