pwc-archive/papers-with-abstracts huggingface dataset:
Reliable Estimation of KL Divergence using a Discriminator in Reproducing Kernel Hilbert Space
Authors: Sandesh Ghimire, Aria Masoomi, Jennifer Dy
Date: 2021-09-29 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Learning Theory

Estimating Kullback Leibler (KL) divergence from samples of two distributions is essential in many machine learning problems. Variational methods using neural network discriminator have been proposed to achieve this task in a scalable manner. However, we noted that most of these methods using neural network discriminators suffer from high fluctuations (variance) in estimates and instability in training. In this paper, we look at this issue from statistical learning theory and function space complexity perspective to understand why this happens and how to solve it. We argue that the cause of these pathologies is lack of control over the complexity of the neural network discriminator function and could be mitigated by controlling it. To achieve this objective, we 1) present a novel construction of the discriminator in the Reproducing Kernel Hilbert Space (RKHS), 2) theoretically relate the error probability bound of the KL estimates to the complexity of the discriminator in the RKHS space, 3) present a scalable way to control the complexity (RKHS norm) of the discriminator for a reliable estimation of KL divergence, and 4) prove the consistency of the proposed estimator. In three different applications of KL divergence : estimation of KL, estimation of mutual information and Variational Bayes, we show that by controlling the complexity as developed in the theory, we are able to reduce the variance of KL estimates and stabilize the training
Active Refinement for Multi-Label Learning: A Pseudo-Label Approach
Authors: Cheng-Yu Hsieh, Wei-I Lin, Miao Xu, Gang Niu, Hsuan-Tien Lin, Masashi Sugiyama
Date: 2021-09-29 00:00:00
Tasks: Active Learning, Multi-Label Learning, Pseudo Label

The goal of multi-label learning (MLL) is to associate a given instance with its relevant labels from a set of concepts. Previous works of MLL mainly focused on the setting where the concept set is assumed to be fixed, while many real-world applications require introducing new concepts into the set to meet new demands. One common need is to refine the original coarse concepts and split them into finer-grained ones, where the refinement process typically begins with limited labeled data for the finer-grained concepts. To address the need, we formalize the problem into a special weakly supervised MLL problem to not only learn the fine-grained concepts efficiently but also allow interactive queries to strategically collect more informative annotations to further improve the classifier. The key idea within our approach is to learn to assign pseudo-labels to the unlabeled entries, and in turn leverage the pseudo-labels to train the underlying classifier and to inform a better query strategy. Experimental results demonstrate that our pseudo-label approach is able to accurately recover the missing ground truth, boosting the prediction performance significantly over the baseline methods and facilitating a competitive active learning strategy.
Data Sharing and Compression for Cooperative Networked Control
Authors: Jiangnan Cheng, Marco Pavone, Sachin Katti, Sandeep Chinchali, Ao Tang
Date: 2021-09-29 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Scheduling

Sharing forecasts of network timeseries data, such as cellular or electricity load patterns, can improve independent control applications ranging from traffic scheduling to power generation. Typically, forecasts are designed without knowledge of a downstream controller's task objective, and thus simply optimize for mean prediction error. However, such task-agnostic representations are often too large to stream over a communication network and do not emphasize salient temporal features for cooperative control. This paper presents a solution to learn succinct, highly-compressed forecasts that are co-designed with a modular controller's task objective. Our simulations with real cellular, Internet-of-Things (IoT), and electricity load data show we can improve a model predictive controller's performance by at least $25\%$ while transmitting $80\%$ less data than the competing method. Further, we present theoretical compression results for a networked variant of the classical linear quadratic regulator (LQR) control problem.
Stroke recovery phenotyping through network trajectory approaches and graph neural networks
Authors: Sanjukta Krishnagopal, Keith Lohse, Robynne Braun
Date: 2021-09-29 00:00:00
Tasks: Clustering

Stroke is a leading cause of neurological injury characterized by impairments in multiple neurological domains including cognition, language, sensory and motor functions. Clinical recovery in these domains is tracked using a wide range of measures that may be continuous, ordinal, interval or categorical in nature, which presents challenges for standard multivariate regression approaches. This has hindered stroke researchers' ability to achieve an integrated picture of the complex time-evolving interactions amongst symptoms. Here we use tools from network science and machine learning that are particularly well-suited to extracting underlying patterns in such data, and may assist in prediction of recovery patterns. To demonstrate the utility of this approach, we analyzed data from the NINDS tPA trial using the Trajectory Profile Clustering (TPC) method to identify distinct stroke recovery patterns for 11 different neurological domains at 5 discrete time points. Our analysis identified 3 distinct stroke trajectory profiles that align with clinically relevant stroke syndromes, characterized both by distinct clusters of symptoms, as well as differing degrees of symptom severity. We then validated our approach using graph neural networks to determine how well our model performed predictively for stratifying patients into these trajectory profiles at early vs. later time points post-stroke. We demonstrate that trajectory profile clustering is an effective method for identifying clinically relevant recovery subtypes in multidimensional longitudinal datasets, and for early prediction of symptom progression subtypes in individual patients. This paper is the first work introducing network trajectory approaches for stroke recovery phenotyping, and is aimed at enhancing the translation of such novel computational approaches for practical clinical application.
An Empirical Study of Accuracy, Fairness, Explainability, Distributional Robustness, and Adversarial Robustness
Authors: Moninder Singh, Gevorg Ghalachyan, Kush R. Varshney, Reginald E. Bryant
Date: 2021-09-29 00:00:00
Tasks: Adversarial Robustness, Fairness

To ensure trust in AI models, it is becoming increasingly apparent that evaluation of models must be extended beyond traditional performance metrics, like accuracy, to other dimensions, such as fairness, explainability, adversarial robustness, and distribution shift. We describe an empirical study to evaluate multiple model types on various metrics along these dimensions on several datasets. Our results show that no particular model type performs well on all dimensions, and demonstrate the kinds of trade-offs involved in selecting models evaluated along multiple dimensions.
A Study of Feature Selection and Extraction Algorithms for Cancer Subtype Prediction
Authors: Vaibhav Sinha, Siladitya Dash, Nazma Naskar, Sk Md Mosaddek Hossain
Date: 2021-09-29 00:00:00
Tasks: Dimensionality Reduction, feature selection

In this work, we study and analyze different feature selection algorithms that can be used to classify cancer subtypes in case of highly varying high-dimensional data. We apply three different feature selection methods on five different types of cancers having two separate omics each. We show that the existing feature selection methods are computationally expensive when applied individually. Instead, we apply these algorithms sequentially which helps in lowering the computational cost and improving the predictive performance. We further show that reducing the number of features using some dimension reduction techniques can improve the performance of machine learning models in some cases. We support our findings through comprehensive data analysis and visualization.
Stock Index Prediction using Cointegration test and Quantile Loss
Authors: Jaeyoung Cheong, Heejoon Lee, Minjung Kang
Date: 2021-09-29 00:00:00
Tasks: Stock Prediction

Recent researches on stock prediction using deep learning methods has been actively studied. This is the task to predict the movement of stock prices in the future based on historical trends. The approach to predicting the movement based solely on the pattern of the historical movement of it on charts, not on fundamental values, is called the Technical Analysis, which can be divided into univariate and multivariate methods in the regression task. According to the latter approach, it is important to select different factors well as inputs to enhance the performance of the model. Moreover, its performance can depend on which loss is used to train the model. However, most studies tend to focus on building the structures of models, not on how to select informative factors as inputs to train them. In this paper, we propose a method that can get better performance in terms of returns when selecting informative factors using the cointegration test and learning the model using quantile loss. We compare the two RNN variants with quantile loss with only five factors obtained through the cointegration test among the entire 15 stock index factors collected in the experiment. The Cumulative return and Sharpe ratio were used to evaluate the performance of trained models. Our experimental results show that our proposed method outperforms the other conventional approaches.
Convolution-Free Waveform Transformers for Multi-Lead ECG Classification
Authors: Annamalai Natarajan, Gregory Boverman, Yale Chang, Corneliu Antonescu, Jonathan Rubin
Date: 2021-09-29 00:00:00
Tasks: Classification, ECG Classification

We present our entry to the 2021 PhysioNet/CinC challenge - a waveform transformer model to detect cardiac abnormalities from ECG recordings. We compare the performance of the waveform transformer model on different ECG-lead subsets using approximately 88,000 ECG recordings from six datasets. In the official rankings, team prna ranked between 9 and 15 on 12, 6, 4, 3 and 2-lead sets respectively. Our waveform transformer model achieved an average challenge metric of 0.47 on the held-out test set across all ECG-lead subsets. Our combined performance across all leads placed us at rank 11 out of 39 officially ranking teams.
Back in Black: A Comparative Evaluation of Recent State-Of-The-Art Black-Box Attacks
Authors: Kaleel Mahmood, Rigel Mahmood, Ethan Rathbun, Marten van Dijk
Date: 2021-09-29 00:00:00
Tasks: BIG-bench Machine Learning

The field of adversarial machine learning has experienced a near exponential growth in the amount of papers being produced since 2018. This massive information output has yet to be properly processed and categorized. In this paper, we seek to help alleviate this problem by systematizing the recent advances in adversarial machine learning black-box attacks since 2019. Our survey summarizes and categorizes 20 recent black-box attacks. We also present a new analysis for understanding the attack success rate with respect to the adversarial model used in each paper. Overall, our paper surveys a wide body of literature to highlight recent attack developments and organizes them into four attack categories: score based attacks, decision based attacks, transfer attacks and non-traditional attacks. Further, we provide a new mathematical framework to show exactly how attack results can fairly be compared.
On the Convergence of Projected Alternating Maximization for Equitable and Optimal Transport
Authors: Minhui Huang, Shiqian Ma, Lifeng Lai
Date: 2021-09-29 00:00:00

This paper studies the equitable and optimal transport (EOT) problem, which has many applications such as fair division problems and optimal transport with multiple agents etc. In the discrete distributions case, the EOT problem can be formulated as a linear program (LP). Since this LP is prohibitively large for general LP solvers, Scetbon \etal \cite{scetbon2021equitable} suggests to perturb the problem by adding an entropy regularization. They proposed a projected alternating maximization algorithm (PAM) to solve the dual of the entropy regularized EOT. In this paper, we provide the first convergence analysis of PAM. A novel rounding procedure is proposed to help construct the primal solution for the original EOT problem. We also propose a variant of PAM by incorporating the extrapolation technique that can numerically improve the performance of PAM. Results in this paper may shed lights on block coordinate (gradient) descent methods for general optimization problems.
Real-Time Multi-Level Neonatal Heart and Lung Sound Quality Assessment for Telehealth Applications
Authors: Ethan Grooby, Chiranjibi Sitaula, Davood Fattahi, Reza Sameni, Kenneth Tan, Lindsay Zhou, Arrabella King, Ashwin Ramanathan, Atul Malhotra, Guy A. Dumont, Faezeh Marzbanrad
Date: 2021-09-29 00:00:00
Tasks: feature selection, Ordinal Classification

Digital stethoscopes in combination with telehealth allow chest sounds to be easily collected and transmitted for remote monitoring and diagnosis. Chest sounds contain important information about a newborn's cardio-respiratory health. However, low-quality recordings complicate the remote monitoring and diagnosis. In this study, a new method is proposed to objectively and automatically assess heart and lung signal quality on a 5-level scale in real-time and to assess the effect of signal quality on vital sign estimation. For the evaluation, a total of 207 10s long chest sounds were taken from 119 preterm and full-term babies. Thirty of the recordings from ten subjects were obtained with synchronous vital signs from the Neonatal Intensive Care Unit (NICU) based on electrocardiogram recordings. As reference, seven annotators independently assessed the signal quality. For automatic quality classification, 400 features were extracted from the chest sounds. After feature selection using minimum redundancy and maximum relevancy algorithm, class balancing, and hyper-parameter optimization, a variety of multi-class and ordinal classification and regression algorithms were trained. Then, heart rate and breathing rate were automatically estimated from the chest sounds using adapted pre-existing methods. The results of subject-wise leave-one-out cross-validation show that the best-performing models had a mean squared error (MSE) of 0.49 and 0.61, and balanced accuracy of 57% and 51% for heart and lung qualities, respectively. The best-performing models for real-time analysis (<200ms) had MSE of 0.459 and 0.67, and balanced accuracy of 57% and 46%, respectively. Our experimental results underscore that increasing the signal quality leads to a reduction in vital sign error, with only high-quality recordings having a mean absolute error of less than 5 beats per minute, as required for clinical usage.
Automated Workers Ergonomic Risk Assessment in Manual Material Handling using sEMG Wearable Sensors and Machine Learning
Authors: Srimantha E. Mudiyanselage, Phuong H. D. Nguyen, Mohammad Sadra Rajabi, Reza Akhavian
Date: 2021-09-27 00:00:00

Manual material handling tasks have the potential to be highly unsafe from an ergonomic viewpoint. Safety inspections to monitor body postures can help mitigate ergonomic risks of material handling. However, the real effect of awkward muscle movements, strains, and excessive forces that may result in an injury may not be identified by external cues. This paper evaluates the ability of surface electromyogram (EMG)-based systems together with machine learning algorithms to automatically detect body movements that may harm muscles in material handling. The analysis utilized a lifting equation developed by the U.S. National Institute for Occupational Safety and Health (NIOSH). This equation determines a Recommended Weight Limit, which suggests the maximum acceptable weight that a healthy worker can lift and carry as well as a Lifting Index value to assess the risk extent. Four different machine learning models, namely Decision Tree, Support Vector Machine, K-Nearest Neighbor, and Random Forest are developed to classify the risk assessments calculated based on the NIOSH lifting equation. The sensitivity of the models to various parameters is also evaluated to find the best performance using each algorithm. Results indicate that Decision Tree models have the potential to predict the risk level with close to 99.35% accuracy.
A Priori Calibration of Transient Kinetics Data via Machine Learning
Authors: M. Ross Kunz, Adam Yonge, Rakesh Batchu, Zongtang Fang, Yixiao Wang, Gregory Yablonsky, Andrew J. Medford, Rebecca Fushimi
Date: 2021-09-27 00:00:00
Tasks: BIG-bench Machine Learning

The temporal analysis of products reactor provides a vast amount of transient kinetic information that may be used to describe a variety of chemical features including the residence time distribution, kinetic coefficients, number of active sites, and the reaction mechanism. However, as with any measurement device, the TAP reactor signal is convoluted with noise. To reduce the uncertainty of the kinetic measurement and any derived parameters or mechanisms, proper preprocessing must be performed prior to any advanced analysis. This preprocessing consists of baseline correction, i.e., a shift in the voltage response, and calibration, i.e., a scaling of the flux response based on prior experiments. The current methodology of preprocessing requires significant user discretion and reliance on previous experiments that may drift over time. Herein we use machine learning techniques combined with physical constraints to convert the raw instrument signal to chemical information. As such, the proposed methodology demonstrates clear benefits over the traditional preprocessing in the calibration of the inert and feed mixture products without need of prior calibration experiments or heuristic input from the user.
Semi-Supervised Text Classification via Self-Pretraining
Authors: Payam Karisani, Negin Karisani
Date: 2021-09-30 00:00:00
Tasks: Pseudo Label, Semi-Supervised Text Classification, Text Classification

We present a neural semi-supervised learning model termed Self-Pretraining. Our model is inspired by the classic self-training algorithm. However, as opposed to self-training, Self-Pretraining is threshold-free, it can potentially update its belief about previously labeled documents, and can cope with the semantic drift problem. Self-Pretraining is iterative and consists of two classifiers. In each iteration, one classifier draws a random set of unlabeled documents and labels them. This set is used to initialize the second classifier, to be further trained by the set of labeled documents. The algorithm proceeds to the next iteration and the classifiers' roles are reversed. To improve the flow of information across the iterations and also to cope with the semantic drift problem, Self-Pretraining employs an iterative distillation process, transfers hypotheses across the iterations, utilizes a two-stage training model, uses an efficient learning rate schedule, and employs a pseudo-label transformation heuristic. We have evaluated our model in three publicly available social media datasets. Our experiments show that Self-Pretraining outperforms the existing state-of-the-art semi-supervised classifiers across multiple settings. Our code is available at https://github.com/p-karisani/self_pretraining.
Multi-granular Legal Topic Classification on Greek Legislation
Authors: Christos Papaloukas, Ilias Chalkidis, Konstantinos Athinaios, Despina-Athanasia Pantazi, Manolis Koubarakis
Date: 2021-09-30 00:00:00
Proceeding: EMNLP (NLLP) 2021 11
Tasks: Classification, text-classification, Text Classification, Topic Classification, Transfer Learning, Word Embeddings

In this work, we study the task of classifying legal texts written in the Greek language. We introduce and make publicly available a novel dataset based on Greek legislation, consisting of more than 47 thousand official, categorized Greek legislation resources. We experiment with this dataset and evaluate a battery of advanced methods and classifiers, ranging from traditional machine learning and RNN-based methods to state-of-the-art Transformer-based methods. We show that recurrent architectures with domain-specific word embeddings offer improved overall performance while being competitive even to transformer-based models. Finally, we show that cutting-edge multilingual and monolingual transformer-based models brawl on the top of the classifiers' ranking, making us question the necessity of training monolingual transfer learning models as a rule of thumb. To the best of our knowledge, this is the first time the task of Greek legal text classification is considered in an open research project, while also Greek is a language with very limited NLP resources in general.
SlovakBERT: Slovak Masked Language Model
Authors: Matúš Pikuliak, Štefan Grivalský, Martin Konôpka, Miroslav Blšták, Martin Tamajka, Viktor Bachratý, Marián Šimko, Pavol Balážik, Michal Trnka, Filip Uhlárik
Date: 2021-09-30 00:00:00
Tasks: Language Modeling, Language Modelling, model, Part-Of-Speech Tagging, Semantic Textual Similarity, Sentiment Analysis

We introduce a new Slovak masked language model called SlovakBERT. This is to our best knowledge the first paper discussing Slovak transformers-based language models. We evaluate our model on several NLP tasks and achieve state-of-the-art results. This evaluation is likewise the first attempt to establish a benchmark for Slovak language models. We publish the masked language model, as well as the fine-tuned models for part-of-speech tagging, sentiment analysis and semantic textual similarity.
A formal model for ledger management systems based on contracts and temporal logic
Authors: Paolo Bottoni, Anna Labella, Remo Pareschi
Date: 2021-09-30 00:00:00
Tasks: Management

A key component of blockchain technology is the ledger, viz., a database that, unlike standard databases, keeps in memory the complete history of past transactions as in a notarial archive for the benefit of any future test. In second-generation blockchains such as Ethereum the ledger is coupled with smart contracts, which enable the automation of transactions associated with agreements between the parties of a financial or commercial nature. The coupling of smart contracts and ledgers provides the technological background for very innovative application areas, such as Decentralized Autonomous Organizations (DAOs), Initial Coin Offerings (ICOs) and Decentralized Finance (DeFi), which propelled blockchains beyond cryptocurrencies that were the only focus of first generation blockchains such as the Bitcoin. However, the currently used implementation of smart contracts as arbitrary programming constructs has made them susceptible to dangerous bugs that can be exploited maliciously and has moved their semantics away from that of legal contracts. We propose here to recompose the split and recover the reliability of databases by formalizing a notion of contract modelled as a finite-state automaton with well-defined computational characteristics derived from an encoding in terms of allocations of resources to actors, as an alternative to the approach based on programming. To complete the work, we use temporal logic as the basis for an abstract query language that is effectively suited to the historical nature of the information kept in the ledger.