pwc-archive/papers-with-abstracts huggingface dataset:
Causal Fairness Assessment of Treatment Allocation with Electronic Health Records
Authors: Linying Zhang, Lauren R. Richter, Yixin Wang, Anna Ostropolets, Noemie Elhadad, David M. Blei, George Hripcsak
Date: 2022-11-21 00:00:00
Tasks: Causal Inference, Decision Making, Fairness

Healthcare continues to grapple with the persistent issue of treatment disparities, sparking concerns regarding the equitable allocation of treatments in clinical practice. While various fairness metrics have emerged to assess fairness in decision-making processes, a growing focus has been on causality-based fairness concepts due to their capacity to mitigate confounding effects and reason about bias. However, the application of causal fairness notions in evaluating the fairness of clinical decision-making with electronic health record (EHR) data remains an understudied domain. This study aims to address the methodological gap in assessing causal fairness of treatment allocation with electronic health records data. We propose a causal fairness algorithm to assess fairness in clinical decision-making. Our algorithm accounts for the heterogeneity of patient populations and identifies potential unfairness in treatment allocation by conditioning on patients who have the same likelihood to benefit from the treatment. We apply this framework to a patient cohort with coronary artery disease derived from an EHR database to evaluate the fairness of treatment decisions. In addition, we investigate the impact of social determinants of health on the assessment of causal fairness of treatment allocation.
HARL: Hierarchical Adaptive Reinforcement Learning Based Auto Scheduler for Neural Networks
Authors: Zining Zhang, Bingsheng He, Zhenjie Zhang
Date: 2022-11-21 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning (RL)

To efficiently perform inference with neural networks, the underlying tensor programs require sufficient tuning efforts before being deployed into production environments. Usually, enormous tensor program candidates need to be sufficiently explored to find the one with the best performance. This is necessary to make the neural network products meet the high demand of real-world applications such as natural language processing, auto-driving, etc. Auto-schedulers are being developed to avoid the need for human intervention. However, due to the gigantic search space and lack of intelligent search guidance, current auto-schedulers require hours to days of tuning time to find the best-performing tensor program for the entire neural network. In this paper, we propose HARL, a reinforcement learning (RL) based auto-scheduler specifically designed for efficient tensor program exploration. HARL uses a hierarchical RL architecture in which learning-based decisions are made at all different levels of search granularity. It also automatically adjusts exploration configurations in real-time for faster performance convergence. As a result, HARL improves the tensor operator performance by 22% and the search speed by 4.3x compared to the state-of-the-art auto-scheduler. Inference performance and search speed are also significantly improved on end-to-end neural networks.
Graph-based Semi-supervised Local Clustering with Few Labeled Nodes
Authors: Zhaiming Shen, Ming-Jun Lai, Sheng Li
Date: 2022-11-20 00:00:00
Tasks: Clustering, Compressive Sensing

Local clustering aims at extracting a local structure inside a graph without the necessity of knowing the entire graph structure. As the local structure is usually small in size compared to the entire graph, one can think of it as a compressive sensing problem where the indices of target cluster can be thought as a sparse solution to a linear system. In this paper, we apply this idea based on two pioneering works under the same framework and propose a new semi-supervised local clustering approach using only few labeled nodes. Our approach improves the existing works by making the initial cut to be the entire graph and hence overcomes a major limitation of the existing works, which is the low quality of initial cut. Extensive experimental results on various datasets demonstrate the effectiveness of our approach.
Frozen Overparameterization: A Double Descent Perspective on Transfer Learning of Deep Neural Networks
Authors: Yehuda Dar, Lorenzo Luzi, Richard G. Baraniuk
Date: 2022-11-20 00:00:00
Tasks: image-classification, Image Classification, Transfer Learning

We study the generalization behavior of transfer learning of deep neural networks (DNNs). We adopt the overparameterization perspective -- featuring interpolation of the training data (i.e., approximately zero train error) and the double descent phenomenon -- to explain the delicate effect of the transfer learning setting on generalization performance. We study how the generalization behavior of transfer learning is affected by the dataset size in the source and target tasks, the number of transferred layers that are kept frozen in the target DNN training, and the similarity between the source and target tasks. We show that the test error evolution during the target DNN training has a more significant double descent effect when the target training dataset is sufficiently large. In addition, a larger source training dataset can yield a slower target DNN training. Moreover, we demonstrate that the number of frozen layers can determine whether the transfer learning is effectively underparameterized or overparameterized and, in turn, this may induce a freezing-wise double descent phenomenon that determines the relative success or failure of learning. Also, we show that the double descent phenomenon may make a transfer from a less related source task better than a transfer from a more related source task. We establish our results using image classification experiments with the ResNet, DenseNet and the vision transformer (ViT) architectures.
Learning Nonlinear Couplings in Network of Agents from a Single Sample Trajectory
Authors: Arash Amini, Qiyu Sun, Nader Motee
Date: 2022-11-20 00:00:00

We consider a class of stochastic dynamical networks whose governing dynamics can be modeled using a coupling function. It is shown that the dynamics of such networks can generate geometrically ergodic trajectories under some reasonable assumptions. We show that a general class of coupling functions can be learned using only one sample trajectory from the network. This is practically plausible as in numerous applications it is desired to run an experiment only once but for a longer period of time, rather than repeating the same experiment multiple times from different initial conditions. Building upon ideas from the concentration inequalities for geometrically ergodic Markov chains, we formulate several results about the convergence of the empirical estimator to the true coupling function. Our theoretical findings are supported by extensive simulation results.
Deep learning delay coordinate dynamics for chaotic attractors from partial observable data
Authors: Charles D. Young, Michael D. Graham
Date: 2022-11-20 00:00:00
Tasks: Time Series, Time Series Analysis

A common problem in time series analysis is to predict dynamics with only scalar or partial observations of the underlying dynamical system. For data on a smooth compact manifold, Takens theorem proves a time delayed embedding of the partial state is diffeomorphic to the attractor, although for chaotic and highly nonlinear systems learning these delay coordinate mappings is challenging. We utilize deep artificial neural networks (ANNs) to learn discrete discrete time maps and continuous time flows of the partial state. Given training data for the full state, we also learn a reconstruction map. Thus, predictions of a time series can be made from the current state and several previous observations with embedding parameters determined from time series analysis. The state space for time evolution is of comparable dimension to reduced order manifold models. These are advantages over recurrent neural network models, which require a high dimensional internal state or additional memory terms and hyperparameters. We demonstrate the capacity of deep ANNs to predict chaotic behavior from a scalar observation on a manifold of dimension three via the Lorenz system. We also consider multivariate observations on the Kuramoto-Sivashinsky equation, where the observation dimension required for accurately reproducing dynamics increases with the manifold dimension via the spatial extent of the system.
Simultaneously Learning Robust Audio Embeddings and balanced Hash codes for Query-by-Example
Authors: Anup Singh, Kris Demuynck, Vipul Arora
Date: 2022-11-20 00:00:00
Tasks: Retrieval, Self-Supervised Learning

Audio fingerprinting systems must efficiently and robustly identify query snippets in an extensive database. To this end, state-of-the-art systems use deep learning to generate compact audio fingerprints. These systems deploy indexing methods, which quantize fingerprints to hash codes in an unsupervised manner to expedite the search. However, these methods generate imbalanced hash codes, leading to their suboptimal performance. Therefore, we propose a self-supervised learning framework to compute fingerprints and balanced hash codes in an end-to-end manner to achieve both fast and accurate retrieval performance. We model hash codes as a balanced clustering process, which we regard as an instance of the optimal transport problem. Experimental results indicate that the proposed approach improves retrieval efficiency while preserving high accuracy, particularly at high distortion levels, compared to the competing methods. Moreover, our system is efficient and scalable in computational load and memory storage.
Convolutional Filtering on Sampled Manifolds
Authors: Zhiyang Wang, Luana Ruiz, Alejandro Ribeiro
Date: 2022-11-20 00:00:00

The increasing availability of geometric data has motivated the need for information processing over non-Euclidean domains modeled as manifolds. The building block for information processing architectures with desirable theoretical properties such as invariance and stability is convolutional filtering. Manifold convolutional filters are defined from the manifold diffusion sequence, constructed by successive applications of the Laplace-Beltrami operator to manifold signals. However, the continuous manifold model can only be accessed by sampling discrete points and building an approximate graph model from the sampled manifold. Effective linear information processing on the manifold requires quantifying the error incurred when approximating manifold convolutions with graph convolutions. In this paper, we derive a non-asymptotic error bound for this approximation, showing that convolutional filtering on the sampled manifold converges to continuous manifold filtering. Our findings are further demonstrated empirically on a problem of navigation control.
Heterogenous Ensemble of Models for Molecular Property Prediction
Authors: Sajad Darabi, Shayan Fazeli, Jiwei Liu, Alexandre Milesi, Pawel Morkisz, Jean-François Puget, Gilberto Titericz
Date: 2022-11-20 00:00:00
Tasks: Molecular Property Prediction, Prediction, Property Prediction, valid

Previous works have demonstrated the importance of considering different modalities on molecules, each of which provide a varied granularity of information for downstream property prediction tasks. Our method combines variants of the recent TransformerM architecture with Transformer, GNN, and ResNet backbone architectures. Models are trained on the 2D data, 3D data, and image modalities of molecular graphs. We ensemble these models with a HuberRegressor. The models are trained on 4 different train/validation splits of the original train + valid datasets. This yields a winning solution to the 2\textsuperscript{nd} edition of the OGB Large-Scale Challenge (2022) on the PCQM4Mv2 molecular property prediction dataset. Our proposed method achieves a test-challenge MAE of $0.0723$ and a validation MAE of $0.07145$. Total inference time for our solution is less than 2 hours. We open-source our code at https://github.com/jfpuget/NVIDIA-PCQM4Mv2.
Safe Reinforcement Learning using Data-Driven Predictive Control
Authors: Mahmoud Selim, Amr Alanwar, M. Watheq El-Kharashi, Hazem M. Abbas, Karl H. Johansson
Date: 2022-11-20 00:00:00
Tasks: continuous-control, Continuous Control, Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Safe Reinforcement Learning

Reinforcement learning (RL) algorithms can achieve state-of-the-art performance in decision-making and continuous control tasks. However, applying RL algorithms on safety-critical systems still needs to be well justified due to the exploration nature of many RL algorithms, especially when the model of the robot and the environment are unknown. To address this challenge, we propose a data-driven safety layer that acts as a filter for unsafe actions. The safety layer uses a data-driven predictive controller to enforce safety guarantees for RL policies during training and after deployment. The RL agent proposes an action that is verified by computing the data-driven reachability analysis. If there is an intersection between the reachable set of the robot using the proposed action, we call the data-driven predictive controller to find the closest safe action to the proposed unsafe action. The safety layer penalizes the RL agent if the proposed action is unsafe and replaces it with the closest safe one. In the simulation, we show that our method outperforms state-of-the-art safe RL methods on the robotics navigation problem for a Turtlebot 3 in Gazebo and a quadrotor in Unreal Engine 4 (UE4).
Machine Learning Methods for Anomaly Detection in Nuclear Power Plant Power Transformers
Authors: Iurii Katser, Dmitriy Raspopov, Vyacheslav Kozitsin, Maxim Mezhov
Date: 2022-11-20 00:00:00
Tasks: Anomaly Detection, Time Series Analysis

Power transformers are an important component of a nuclear power plant (NPP). Currently, the NPP operates a lot of power transformers with extended service life, which exceeds the designated 25 years. Due to the extension of the service life, the task of monitoring the technical condition of power transformers becomes urgent. An important method for monitoring power transformers is Chromatographic Analysis of Dissolved Gas. It is based on the principle of controlling the concentration of gases dissolved in transformer oil. The appearance of almost any type of defect in equipment is accompanied by the formation of gases that dissolve in oil, and specific types of defects generate their gases in different quantities. At present, at NPPs, the monitoring systems for transformer equipment use predefined control limits for the concentration of dissolved gases in the oil. This study describes the stages of developing an algorithm to detect defects and faults in transformers automatically using machine learning and data analysis methods. Among machine learning models, we trained Logistic Regression, Decision Trees, Random Forest, Gradient Boosting, Neural Networks. The best of them were then combined into an ensemble (StackingClassifier) showing F1-score of 0.974 on a test sample. To develop mathematical models, we used data on the state of transformers, containing time series with values of gas concentrations (H2, CO, C2H4, C2H2). The datasets were labeled and contained four operating modes: normal mode, partial discharge, low energy discharge, low-temperature overheating.
Turning Silver into Gold: Domain Adaptation with Noisy Labels for Wearable Cardio-Respiratory Fitness Prediction
Authors: Yu Wu, Dimitris Spathis, Hong Jia, Ignacio Perez-Pozuelo, Tomas I. Gonzales, Soren Brage, Nicholas Wareham, Cecilia Mascolo
Date: 2022-11-20 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Deep learning models have shown great promise in various healthcare applications. However, most models are developed and validated on small-scale datasets, as collecting high-quality (gold-standard) labels for health applications is often costly and time-consuming. As a result, these models may suffer from overfitting and not generalize well to unseen data. At the same time, an extensive amount of data with imprecise labels (silver-standard) is starting to be generally available, as collected from inexpensive wearables like accelerometers and electrocardiography sensors. These currently underutilized datasets and labels can be leveraged to produce more accurate clinical models. In this work, we propose UDAMA, a novel model with two key components: Unsupervised Domain Adaptation and Multi-discriminator Adversarial training, which leverage noisy data from source domain (the silver-standard dataset) to improve gold-standard modeling. We validate our framework on the challenging task of predicting lab-measured maximal oxygen consumption (VO$_{2}$max), the benchmark metric of cardio-respiratory fitness, using free-living wearable sensor data from two cohort studies as inputs. Our experiments show that the proposed framework achieves the best performance of corr = 0.665 $\pm$ 0.04, paving the way for accurate fitness estimation at scale.
Enhancing Intra-class Information Extraction for Heterophilous Graphs: One Neural Architecture Search Approach
Authors: Lanning Wei, Zhiqiang He, Huan Zhao, Quanming Yao
Date: 2022-11-20 00:00:00
Tasks: Graph Representation Learning, Neural Architecture Search, Node Classification, Representation Learning

In recent years, Graph Neural Networks (GNNs) have been popular in graph representation learning which assumes the homophily property, i.e., the connected nodes have the same label or have similar features. However, they may fail to generalize into the heterophilous graphs which in the low/medium level of homophily. Existing methods tend to address this problem by enhancing the intra-class information extraction, i.e., either by designing better GNNs to improve the model effectiveness, or re-designing the graph structures to incorporate more potential intra-class nodes from distant hops. Despite the success, we observe two aspects that can be further improved: (a) enhancing the ego feature information extraction from node itself which is more reliable in extracting the intra-class information; (b) designing node-wise GNNs can better adapt to the nodes with different homophily ratios. In this paper, we propose a novel method IIE-GNN (Intra-class Information Enhanced Graph Neural Networks) to achieve two improvements. A unified framework is proposed based on the literature, in which the intra-class information from the node itself and neighbors can be extracted based on seven carefully designed blocks. With the help of neural architecture search (NAS), we propose a novel search space based on the framework, and then provide an architecture predictor to design GNNs for each node. We further conduct experiments to show that IIE-GNN can improve the model performance by designing node-wise GNNs to enhance intra-class information extraction.
Federated deep transfer learning for EEG decoding using multiple BCI tasks
Authors: Xiaoxi Wei, A. Aldo Faisal
Date: 2022-11-20 00:00:00
Tasks: Decoder, EEG, Eeg Decoding, Electroencephalogram (EEG), Privacy Preserving, Transfer Learning

Deep learning has been successful in BCI decoding. However, it is very data-hungry and requires pooling data from multiple sources. EEG data from various sources decrease the decoding performance due to negative transfer. Recently, transfer learning for EEG decoding has been suggested as a remedy and become subject to recent BCI competitions (e.g. BEETL), but there are two complications in combining data from many subjects. First, privacy is not protected as highly personal brain data needs to be shared (and copied across increasingly tight information governance boundaries). Moreover, BCI data are collected from different sources and are often based on different BCI tasks, which has been thought to limit their reusability. Here, we demonstrate a federated deep transfer learning technique, the Multi-dataset Federated Separate-Common-Separate Network (MF-SCSN) based on our previous work of SCSN, which integrates privacy-preserving properties into deep transfer learning to utilise data sets with different tasks. This framework trains a BCI decoder using different source data sets obtained from different imagery tasks (e.g. some data sets with hands and feet, vs others with single hands and tongue, etc). Therefore, by introducing privacy-preserving transfer learning techniques, we unlock the reusability and scalability of existing BCI data sets. We evaluated our federated transfer learning method on the NeurIPS 2021 BEETL competition BCI task. The proposed architecture outperformed the baseline decoder by 3%. Moreover, compared with the baseline and other transfer learning algorithms, our method protects the privacy of the brain data from different data centres.
When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration Method
Authors: Manyi Zhang, Xuyang Zhao, Jun Yao, Chun Yuan, Weiran Huang
Date: 2022-11-20 00:00:00
Proceeding: ICCV 2023 1
Tasks: Contrastive Learning, Learning with noisy labels, Representation Learning

Real-world large-scale datasets are both noisily labeled and class-imbalanced. The issues seriously hurt the generalization of trained models. It is hence significant to address the simultaneous incorrect labeling and class-imbalance, i.e., the problem of learning with noisy labels on long-tailed data. Previous works develop several methods for the problem. However, they always rely on strong assumptions that are invalid or hard to be checked in practice. In this paper, to handle the problem and address the limitations of prior works, we propose a representation calibration method RCAL. Specifically, RCAL works with the representations extracted by unsupervised contrastive learning. We assume that without incorrect labeling and class imbalance, the representations of instances in each class conform to a multivariate Gaussian distribution, which is much milder and easier to be checked. Based on the assumption, we recover underlying representation distributions from polluted ones resulting from mislabeled and class-imbalanced data. Additional data points are then sampled from the recovered distributions to help generalization. Moreover, during classifier training, representation learning takes advantage of representation robustness brought by contrastive learning, which further improves the classifier performance. We derive theoretical results to discuss the effectiveness of our representation calibration. Experiments on multiple benchmarks justify our claims and confirm the superiority of the proposed method.
MEESO: A Multi-objective End-to-End Self-Optimized Approach for Automatically Building Deep Learning Models
Authors: Thanh Phuong Pham
Date: 2022-11-20 00:00:00
Tasks: Deep Learning, Diversity, Model Selection

Deep learning has been widely used in various applications from different fields such as computer vision, natural language processing, etc. However, the training models are often manually developed via many costly experiments. This manual work usually requires substantial computing resources, time, and experience. To simplify the use of deep learning and alleviate human effort, automated deep learning has emerged as a potential tool that releases the burden for both users and researchers. Generally, an automatic approach should support the diversity of model selection and the evaluation should allow users to decide upon their demands. To that end, we propose a multi-objective end-to-end self-optimized approach for constructing deep learning models automatically. Experimental results on well-known datasets such as MNIST, Fashion, and Cifar10 show that our algorithm can discover various competitive models compared with the state-of-the-art approach. In addition, our approach also introduces multi-objective trade-off solutions for both accuracy and uncertainty metrics for users to make better decisions.