pwc-archive/papers-with-abstracts huggingface dataset:
Stochastic Model Predictive Control Utilizing Bayesian Neural Networks
Authors: J. Pohlodek, H. Alsmeier, B. Morabito, C. Schlauch, A. Savchenko, R. Findeisen
Date: 2023-03-25 00:00:00
Tasks: Gaussian Processes, model, Model Predictive Control

Integrating measurements and historical data can enhance control systems through learning-based techniques, but ensuring performance and safety is challenging. Robust model predictive control strategies, like stochastic model predictive control, can address this by accounting for uncertainty. Gaussian processes are often used but have limitations with larger models and data sets. We explore Bayesian neural networks for stochastic learning-assisted control, comparing their performance to Gaussian processes on a wastewater treatment plant model. Results show Bayesian neural networks achieve similar performance, highlighting their potential as an alternative for control designs, particularly when handling extensive data sets.
Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey
Authors: Guangyin Jin, Yuxuan Liang, Yuchen Fang, Zezhi Shao, Jincai Huang, Junbo Zhang, Yu Zheng
Date: 2023-03-25 00:00:00
Tasks: Management

With recent advances in sensing technologies, a myriad of spatio-temporal data has been generated and recorded in smart cities. Forecasting the evolution patterns of spatio-temporal data is an important yet demanding aspect of urban computing, which can enhance intelligent management decisions in various fields, including transportation, environment, climate, public safety, healthcare, and others. Traditional statistical and deep learning methods struggle to capture complex correlations in urban spatio-temporal data. To this end, Spatio-Temporal Graph Neural Networks (STGNN) have been proposed, achieving great promise in recent years. STGNNs enable the extraction of complex spatio-temporal dependencies by integrating graph neural networks (GNNs) and various temporal learning methods. In this manuscript, we provide a comprehensive survey on recent progress on STGNN technologies for predictive learning in urban computing. Firstly, we provide a brief introduction to the construction methods of spatio-temporal graph data and the prevalent deep-learning architectures used in STGNNs. We then sort out the primary application domains and specific predictive learning tasks based on existing literature. Afterward, we scrutinize the design of STGNNs and their combination with some advanced technologies in recent years. Finally, we conclude the limitations of existing research and suggest potential directions for future work.
Autoregressive Conditional Neural Processes
Authors: Wessel P. Bruinsma, Stratis Markou, James Requiema, Andrew Y. K. Foong, Tom R. Andersson, Anna Vaughan, Anthony Buonomo, J. Scott Hosking, Richard E. Turner
Date: 2023-03-25 00:00:00
Tasks: Meta-Learning

Conditional neural processes (CNPs; Garnelo et al., 2018a) are attractive meta-learning models which produce well-calibrated predictions and are trainable via a simple maximum likelihood procedure. Although CNPs have many advantages, they are unable to model dependencies in their predictions. Various works propose solutions to this, but these come at the cost of either requiring approximate inference or being limited to Gaussian predictions. In this work, we instead propose to change how CNPs are deployed at test time, without any modifications to the model or training procedure. Instead of making predictions independently for every target point, we autoregressively define a joint predictive distribution using the chain rule of probability, taking inspiration from the neural autoregressive density estimator (NADE) literature. We show that this simple procedure allows factorised Gaussian CNPs to model highly dependent, non-Gaussian predictive distributions. Perhaps surprisingly, in an extensive range of tasks with synthetic and real data, we show that CNPs in autoregressive (AR) mode not only significantly outperform non-AR CNPs, but are also competitive with more sophisticated models that are significantly more computationally expensive and challenging to train. This performance is remarkable given that AR CNPs are not trained to model joint dependencies. Our work provides an example of how ideas from neural distribution estimation can benefit neural processes, and motivates research into the AR deployment of other neural process models.
Verifying Properties of Tsetlin Machines
Authors: Emilia Przybysz, Bimal Bhattarai, Cosimo Persia, Ana Ozaki, Ole-Christoffer Granmo, Jivitesh Sharma
Date: 2023-03-25 00:00:00
Tasks: Adversarial Robustness, Interpretable Machine Learning, Sentiment Analysis, Sentiment Classification

Tsetlin Machines (TsMs) are a promising and interpretable machine learning method which can be applied for various classification tasks. We present an exact encoding of TsMs into propositional logic and formally verify properties of TsMs using a SAT solver. In particular, we introduce in this work a notion of similarity of machine learning models and apply our notion to check for similarity of TsMs. We also consider notions of robustness and equivalence from the literature and adapt them for TsMs. Then, we show the correctness of our encoding and provide results for the properties: adversarial robustness, equivalence, and similarity of TsMs. In our experiments, we employ the MNIST and IMDB datasets for (respectively) image and sentiment classification. We discuss the results for verifying robustness obtained with TsMs with those in the literature obtained with Binarized Neural Networks on MNIST.
No more Reviewer #2: Subverting Automatic Paper-Reviewer Assignment using Adversarial Learning
Authors: Thorsten Eisenhofer, Erwin Quiring, Jonas Möller, Doreen Riepel, Thorsten Holz, Konrad Rieck
Date: 2023-03-25 00:00:00
Tasks: Topic Models

The number of papers submitted to academic conferences is steadily rising in many scientific disciplines. To handle this growth, systems for automatic paper-reviewer assignments are increasingly used during the reviewing process. These systems use statistical topic models to characterize the content of submissions and automate the assignment to reviewers. In this paper, we show that this automation can be manipulated using adversarial learning. We propose an attack that adapts a given paper so that it misleads the assignment and selects its own reviewers. Our attack is based on a novel optimization strategy that alternates between the feature space and problem space to realize unobtrusive changes to the paper. To evaluate the feasibility of our attack, we simulate the paper-reviewer assignment of an actual security conference (IEEE S&P) with 165 reviewers on the program committee. Our results show that we can successfully select and remove reviewers without access to the assignment system. Moreover, we demonstrate that the manipulated papers remain plausible and are often indistinguishable from benign submissions.
Heat flux for semi-local machine-learning potentials
Authors: Marcel F. Langer, Florian Knoop, Christian Carbogno, Matthias Scheffler, Matthias Rupp
Date: 2023-03-25 00:00:00
Tasks: Computational Efficiency

The Green-Kubo (GK) method is a rigorous framework for heat transport simulations in materials. However, it requires an accurate description of the potential-energy surface and carefully converged statistics. Machine-learning potentials can achieve the accuracy of first-principles simulations while allowing to reach well beyond their simulation time and length scales at a fraction of the cost. In this paper, we explain how to apply the GK approach to the recent class of message-passing machine-learning potentials, which iteratively consider semi-local interactions beyond the initial interaction cutoff. We derive an adapted heat flux formulation that can be implemented using automatic differentiation without compromising computational efficiency. The approach is demonstrated and validated by calculating the thermal conductivity of zirconium dioxide across temperatures.
Task-Attentive Transformer Architecture for Continual Learning of Vision-and-Language Tasks Using Knowledge Distillation
Authors: Yuliang Cai, Jesse Thomason, Mohammad Rostami
Date: 2023-03-25 00:00:00
Tasks: Continual Learning, Knowledge Distillation, Transfer Learning

The size and the computational load of fine-tuning large-scale pre-trained neural network are becoming two major obstacles in adopting machine learning in many applications. Continual learning (CL) can serve as a remedy through enabling knowledge-transfer across sequentially arriving tasks which relaxes the need to fine-tune all network weights from scratch. However, existing CL algorithms primarily consider learning unimodal vision-only or language-only tasks. We develop a transformer-based CL architecture for learning bimodal vision-and-language tasks based on increasing the number of the learnable parameters dynamically and using knowledge distillation. The new additional parameters are used to specialize the network for each task. Our approach enables sharing information between the tasks while addressing the challenge of catastrophic forgetting. Our approach is scalable learning to a large number of tasks because it requires little memory and time overhead. Our model reaches state-of-the-art performance on challenging vision-and-language tasks.
Spatially-aware station based car-sharing demand prediction
Authors: Dominik J. Mühlematter, Nina Wiedemann, Yanan Xin, Martin Raubal
Date: 2023-03-25 00:00:00
Tasks: Prediction

In recent years, car-sharing services have emerged as viable alternatives to private individual mobility, promising more sustainable and resource-efficient, but still comfortable transportation. Research on short-term prediction and optimization methods has improved operations and fleet control of car-sharing services; however, long-term projections and spatial analysis are sparse in the literature. We propose to analyze the average monthly demand in a station-based car-sharing service with spatially-aware learning algorithms that offer high predictive performance as well as interpretability. Our study utilizes a rich set of socio-demographic, location-based (e.g., POIs), and car-sharing-specific features as input, extracted from a large proprietary car-sharing dataset and publicly available datasets. We first compare the performance of different modeling approaches and find that a global Random Forest with geo-coordinates as part of input features achieves the highest predictive performance with an R-squared score of 0.87 on test data. While a local linear model, Geographically Weighted Regression, performs almost on par in terms of out-of-sample prediction accuracy. We further leverage the models to identify spatial and socio-demographic drivers of car-sharing demand. An analysis of the Random Forest via SHAP values, as well as the coefficients of GWR and MGWR models, reveals that besides population density and the car-sharing supply, other spatial features such as surrounding POIs play a major role. In addition, MGWR yields exciting insights into the multiscale heterogeneous spatial distributions of factors influencing car-sharing behaviour. Together, our study offers insights for selecting effective and interpretable methods for diagnosing and planning the placement of car-sharing stations.
Deep Linear Discriminant Analysis with Variation for Polycystic Ovary Syndrome Classification
Authors: Raunak Joshi, Abhishek Gupta, Himanshu Soni, Ronald Laban
Date: 2023-03-25 00:00:00
Tasks: Deep Learning, Dimensionality Reduction

The polycystic ovary syndrome diagnosis is a problem that can be leveraged using prognostication based learning procedures. Many implementations of PCOS can be seen with Machine Learning but the algorithms have certain limitations in utilizing the processing power graphical processing units. The simple machine learning algorithms can be improved with advanced frameworks using Deep Learning. The Linear Discriminant Analysis is a linear dimensionality reduction algorithm for classification that can be boosted in terms of performance using deep learning with Deep LDA, a transformed version of the traditional LDA. In this result oriented paper we present the Deep LDA implementation with a variation for prognostication of PCOS.
Hybrid Fuzzy-Crisp Clustering Algorithm: Theory and Experiments
Authors: Akira R. Kinjo, Daphne Teck Ching Lai
Date: 2023-03-25 00:00:00
Tasks: Clustering

With the membership function being strictly positive, the conventional fuzzy c-means clustering method sometimes causes imbalanced influence when clusters of vastly different sizes exist. That is, an outstandingly large cluster drags to its center all the other clusters, however far they are separated. To solve this problem, we propose a hybrid fuzzy-crisp clustering algorithm based on a target function combining linear and quadratic terms of the membership function. In this algorithm, the membership of a data point to a cluster is automatically set to exactly zero if the data point is ``sufficiently'' far from the cluster center. In this paper, we present a new algorithm for hybrid fuzzy-crisp clustering along with its geometric interpretation. The algorithm is tested on twenty simulated data generated and five real-world datasets from the UCI repository and compared with conventional fuzzy and crisp clustering methods. The proposed algorithm is demonstrated to outperform the conventional methods on imbalanced datasets and can be competitive on more balanced datasets.
Hierarchical Multi-Agent Multi-Armed Bandit for Resource Allocation in Multi-LEO Satellite Constellation Networks
Authors: Li-Hsiang Shen, Yun Ho, Kai-Ten Feng, Lie-Liang Yang, Sau-Hsuan Wu, Jen-Ming Wu
Date: 2023-03-25 00:00:00
Tasks: Management

Low Earth orbit (LEO) satellite constellation is capable of providing global coverage area with high-rate services in the next sixth-generation (6G) non-terrestrial network (NTN). Due to limited onboard resources of operating power, beams, and channels, resilient and efficient resource management has become compellingly imperative under complex interference cases. However, different from conventional terrestrial base stations, LEO is deployed at considerable height and under high mobility, inducing substantially long delay and interference during transmission. As a result, acquiring the accurate channel state information between LEOs and ground users is challenging. Therefore, we construct a framework with a two-way transmission under unknown channel information and no data collected at long-delay ground gateway. In this paper, we propose hierarchical multi-agent multi-armed bandit resource allocation for LEO constellation (mmRAL) by appropriately assigning available radio resources. LEOs are considered as collaborative multiple macro-agents attempting unknown trials of various actions of micro-agents of respective resources, asymptotically achieving suitable allocation with only throughput information. In simulations, we evaluate mmRAL in various cases of LEO deployment, serving numbers of users and LEOs, hardware cost and outage probability. Benefited by efficient and resilient allocation, the proposed mmRAL system is capable of operating in homogeneous or heterogeneous orbital planes or constellations, achieving the highest throughput performance compared to the existing benchmarks in open literature.
Causal Image Synthesis of Brain MR in 3D
Authors: Yujia Li, Jiong Shi, S. Kevin Zhou
Date: 2023-03-25 00:00:00
Tasks: counterfactual, Counterfactual Reasoning, Decision Making, Image Generation

Clinical decision making requires counterfactual reasoning based on a factual medical image and thus necessitates causal image synthesis. To this end, we present a novel method for modeling the causality between demographic variables, clinical indices and brain MR images for Alzheimer's Diseases. Specifically, we leverage a structural causal model to depict the causality and a styled generator to synthesize the image. Furthermore, as a crucial step to reduce modeling complexity and make learning tractable, we propose the use of low dimensional latent feature representation of a high-dimensional 3D image, together with exogenous noise, to build causal relationship between the image and non image variables. We experiment the proposed method based on 1586 subjects and 3683 3D images and synthesize counterfactual brain MR images intervened on certain attributes, such as age, brain volume and cognitive test score. Quantitative metrics and qualitative evaluation of counterfactual images demonstrates the superiority of our generated images.
Adaptive Bi-Recommendation and Self-Improving Network for Heterogeneous Domain Adaptation-Assisted IoT Intrusion Detection
Authors: Jiashu Wu, Yang Wang, Hao Dai, Chengzhong Xu, Kenneth B. Kent
Date: 2023-03-25 00:00:00
Tasks: Domain Adaptation, Intrusion Detection, Pseudo Label, Recommendation Systems, Transfer Learning

As Internet of Things devices become prevalent, using intrusion detection to protect IoT from malicious intrusions is of vital importance. However, the data scarcity of IoT hinders the effectiveness of traditional intrusion detection methods. To tackle this issue, in this paper, we propose the Adaptive Bi-Recommendation and Self-Improving Network (ABRSI) based on unsupervised heterogeneous domain adaptation (HDA). The ABRSI transfers enrich intrusion knowledge from a data-rich network intrusion source domain to facilitate effective intrusion detection for data-scarce IoT target domains. The ABRSI achieves fine-grained intrusion knowledge transfer via adaptive bi-recommendation matching. Matching the bi-recommendation interests of two recommender systems and the alignment of intrusion categories in the shared feature space form a mutual-benefit loop. Besides, the ABRSI uses a self-improving mechanism, autonomously improving the intrusion knowledge transfer from four ways. A hard pseudo label voting mechanism jointly considers recommender system decision and label relationship information to promote more accurate hard pseudo label assignment. To promote diversity and target data participation during intrusion knowledge transfer, target instances failing to be assigned with a hard pseudo label will be assigned with a probabilistic soft pseudo label, forming a hybrid pseudo-labelling strategy. Meanwhile, the ABRSI also makes soft pseudo-labels globally diverse and individually certain. Finally, an error knowledge learning mechanism is utilised to adversarially exploit factors that causes detection ambiguity and learns through both current and previous error knowledge, preventing error knowledge forgetfulness. Holistically, these mechanisms form the ABRSI model that boosts IoT intrusion detection accuracy via HDA-assisted intrusion knowledge transfer.
Natural Language-Based Synthetic Data Generation for Cluster Analysis
Authors: Michael J. Zellinger, Peter Bühlmann
Date: 2023-03-24 00:00:00
Tasks: Synthetic Data Generation

Cluster analysis relies on effective benchmarks for evaluating and comparing different algorithms. Simulation studies on synthetic data are popular because important features of the data sets, such as the overlap between clusters, or the variation in cluster shapes, can be effectively varied. Unfortunately, creating evaluation scenarios is often laborious, as practitioners must translate higher-level scenario descriptions like "clusters with very different shapes" into lower-level geometric parameters such as cluster centers, covariance matrices, etc. To make benchmarks more convenient and informative, we propose synthetic data generation based on direct specification of high-level scenarios, either through verbal descriptions or high-level geometric parameters. Our open-source Python package repliclust implements this workflow, making it easy to set up interpretable and reproducible benchmarks for cluster analysis. A demo of data generation from verbal inputs is available at https://demo.repliclust.org.
Efficient Lipschitzian Global Optimization of Hölder Continuous Multivariate Functions
Authors: Kaan Gokcesu, Hakan Gokcesu
Date: 2023-03-24 00:00:00
Tasks: global-optimization

This study presents an effective global optimization technique designed for multivariate functions that are H\"older continuous. Unlike traditional methods that construct lower bounding proxy functions, this algorithm employs a predetermined query creation rule that makes it computationally superior. The algorithm's performance is assessed using the average or cumulative regret, which also implies a bound for the simple regret and reflects the overall effectiveness of the approach. The results show that with appropriate parameters the algorithm attains an average regret bound of $O(T^{-\frac{\alpha}{n}})$ for optimizing a H\"older continuous target function with H\"older exponent $\alpha$ in an $n$-dimensional space within a given time horizon $T$. We demonstrate that this bound is minimax optimal.
Applications of Gaussian Processes at Extreme Lengthscales: From Molecules to Black Holes
Authors: Ryan-Rhys Griffiths
Date: 2023-03-24 00:00:00
Tasks: Bayesian Optimisation, Gaussian Processes, scientific discovery

In many areas of the observational and experimental sciences data is scarce. Data observation in high-energy astrophysics is disrupted by celestial occlusions and limited telescope time while data derived from laboratory experiments in synthetic chemistry and materials science is time and cost-intensive to collect. On the other hand, knowledge about the data-generation mechanism is often available in the sciences, such as the measurement error of a piece of laboratory apparatus. Both characteristics, small data and knowledge of the underlying physics, make Gaussian processes (GPs) ideal candidates for fitting such datasets. GPs can make predictions with consideration of uncertainty, for example in the virtual screening of molecules and materials, and can also make inferences about incomplete data such as the latent emission signature from a black hole accretion disc. Furthermore, GPs are currently the workhorse model for Bayesian optimisation, a methodology foreseen to be a guide for laboratory experiments in scientific discovery campaigns. The first contribution of this thesis is to use GP modelling to reason about the latent emission signature from the Seyfert galaxy Markarian 335, and by extension, to reason about the applicability of various theoretical models of black hole accretion discs. The second contribution is to extend the GP framework to molecular and chemical reaction representations and to provide an open-source software library to enable the framework to be used by scientists. The third contribution is to leverage GPs to discover novel and performant photoswitch molecules. The fourth contribution is to introduce a Bayesian optimisation scheme capable of modelling aleatoric uncertainty to facilitate the identification of material compositions that possess intrinsic robustness to large scale fabrication processes.
Feature Space Sketching for Logistic Regression
Authors: Gregory Dexter, Rajiv Khanna, Jawad Raheel, Petros Drineas
Date: 2023-03-24 00:00:00
Tasks: Dimensionality Reduction, feature selection, regression

We present novel bounds for coreset construction, feature selection, and dimensionality reduction for logistic regression. All three approaches can be thought of as sketching the logistic regression inputs. On the coreset construction front, we resolve open problems from prior work and present novel bounds for the complexity of coreset construction methods. On the feature selection and dimensionality reduction front, we initiate the study of forward error bounds for logistic regression. Our bounds are tight up to constant factors and our forward error bounds can be extended to Generalized Linear Models.