pwc-archive/papers-with-abstracts huggingface dataset:
On uniqueness in structured model learning
Authors: Martin Holler, Erion Morina
Date: 2024-10-29 00:00:00
Tasks: model

This paper addresses the problem of uniqueness in learning physical laws for systems of partial differential equations (PDEs). Contrary to most existing approaches, it considers a framework of structured model learning, where existing, approximately correct physical models are augmented with components that are learned from data. The main result of the paper is a uniqueness result that covers a large class of PDEs and a suitable class of neural networks used for approximating the unknown model components. The uniqueness result shows that, in the idealized setting of full, noiseless measurements, a unique identification of the unknown model components is possible as regularization-minimizing solution of the PDE system. Furthermore, the paper provides a convergence result showing that model components learned on the basis of incomplete, noisy measurements approximate the ground truth model component in the limit. These results are possible under specific properties of the approximating neural networks and due to a dedicated choice of regularization. With this, a practical contribution of this analytic paper is to provide a class of model learning frameworks different to standard settings where uniqueness can be expected in the limit of full measurements.
Node Regression on Latent Position Random Graphs via Local Averaging
Authors: Martin Gjorgjevski, Nicolas Keriven, Simon Barthelmé, Yohann de Castro
Date: 2024-10-29 00:00:00
Tasks: Graph Regression, Node Regression, Position, regression

Node regression consists in predicting the value of a graph label at a node, given observations at the other nodes. To gain some insight into the performance of various estimators for this task, we perform a theoretical study in a context where the graph is random. Specifically, we assume that the graph is generated by a Latent Position Model, where each node of the graph has a latent position, and the probability that two nodes are connected depend on the distance between the latent positions of the two nodes. In this context, we begin by studying the simplest possible estimator for graph regression, which consists in averaging the value of the label at all neighboring nodes. We show that in Latent Position Models this estimator tends to a Nadaraya Watson estimator in the latent space, and that its rate of convergence is in fact the same. One issue with this standard estimator is that it averages over a region consisting of all neighbors of a node, and that depending on the graph model this may be too much or too little. An alternative consists in first estimating the true distances between the latent positions, then injecting these estimated distances into a classical Nadaraya Watson estimator. This enables averaging in regions either smaller or larger than the typical graph neighborhood. We show that this method can achieve standard nonparametric rates in certain instances even when the graph neighborhood is too large or too small.
Individualised recovery trajectories of patients with impeded mobility, using distance between probability distributions of learnt graphs
Authors: Chuqiao Zhang, Crina Grosan, Dalia Chakrabarty
Date: 2024-10-29 00:00:00
Tasks: Time Series

Patients who are undergoing physical rehabilitation, benefit from feedback that follows from reliable assessment of their cumulative performance attained at a given time. In this paper, we provide a method for the learning of the recovery trajectory of an individual patient, as they undertake exercises as part of their physical therapy towards recovery of their loss of movement ability, following a critical illness. The difference between the Movement Recovery Scores (MRSs) attained by a patient, when undertaking a given exercise routine on successive instances, is given by a statistical distance/divergence between the (posterior) probabilities of random graphs that are Bayesianly learnt using time series data on locations of 20 of the patient's joints, recorded on an e-platform as the patient exercises. This allows for the computation of the MRS on every occasion the patient undertakes this exercise, using which, the recovery trajectory is drawn. We learn each graph as a Random Geometric Graph drawn in a probabilistic metric space, and identify the closed-form marginal posterior of any edge of the graph, given the correlation structure of the multivariate time series data on joint locations. On the basis of our recovery learning, we offer recommendations on the optimal exercise routines for patients with given level of mobility impairment.
On the Robustness of Adversarial Training Against Uncertainty Attacks
Authors: Emanuele Ledda, Giovanni Scodeller, Daniele Angioni, Giorgio Piras, Antonio Emanuele Cinà, Giorgio Fumera, Battista Biggio, Fabio Roli
Date: 2024-10-29 00:00:00
Tasks: Decision Making

In learning problems, the noise inherent to the task at hand hinders the possibility to infer without a certain degree of uncertainty. Quantifying this uncertainty, regardless of its wide use, assumes high relevance for security-sensitive applications. Within these scenarios, it becomes fundamental to guarantee good (i.e., trustworthy) uncertainty measures, which downstream modules can securely employ to drive the final decision-making process. However, an attacker may be interested in forcing the system to produce either (i) highly uncertain outputs jeopardizing the system's availability or (ii) low uncertainty estimates, making the system accept uncertain samples that would instead require a careful inspection (e.g., human intervention). Therefore, it becomes fundamental to understand how to obtain robust uncertainty estimates against these kinds of attacks. In this work, we reveal both empirically and theoretically that defending against adversarial examples, i.e., carefully perturbed samples that cause misclassification, additionally guarantees a more secure, trustworthy uncertainty estimate under common attack scenarios without the need for an ad-hoc defense strategy. To support our claims, we evaluate multiple adversarial-robust models from the publicly available benchmark RobustBench on the CIFAR-10 and ImageNet datasets.
Online Test of a Neural Network Deep Convection Parameterization in ARP-GEM1
Authors: Blanka Balogh, David Saint-Martin, Olivier Geoffroy
Date: 2024-10-29 00:00:00

In this study, we present the integration of a neural network-based parameterization into the global atmospheric model ARP-GEM1, leveraging the Python interface of the OASIS coupler. This approach facilitates the exchange of fields between the Fortran-based ARP-GEM1 model and a Python component responsible for neural network inference. As a proof-of-concept experiment, we trained a neural network to emulate the deep convection parameterization of ARP-GEM1. Using the flexible Fortran/Python interface, we have successfully replaced ARP-GEM1's deep convection scheme with a neural network emulator. To assess the performance of the neural network deep convection scheme, we have run a 5-years ARP-GEM1 simulation using the neural network emulator. The evaluation of averaged fields showed good agreement with output from an ARP-GEM1 simulation using the physics-based deep convection scheme. The Python component was deployed on a separate partition from the general circulation model, using GPUs to increase inference speed of the neural network.
Identifiability Analysis of Linear ODE Systems with Hidden Confounders
Authors: Yuanyuan Wang, Biwei Huang, Wei Huang, Xi Geng, Mingming Gong
Date: 2024-10-29 00:00:00

The identifiability analysis of linear Ordinary Differential Equation (ODE) systems is a necessary prerequisite for making reliable causal inferences about these systems. While identifiability has been well studied in scenarios where the system is fully observable, the conditions for identifiability remain unexplored when latent variables interact with the system. This paper aims to address this gap by presenting a systematic analysis of identifiability in linear ODE systems incorporating hidden confounders. Specifically, we investigate two cases of such systems. In the first case, latent confounders exhibit no causal relationships, yet their evolution adheres to specific functional forms, such as polynomial functions of time $t$. Subsequently, we extend this analysis to encompass scenarios where hidden confounders exhibit causal dependencies, with the causal structure of latent variables described by a Directed Acyclic Graph (DAG). The second case represents a more intricate variation of the first case, prompting a more comprehensive identifiability analysis. Accordingly, we conduct detailed identifiability analyses of the second system under various observation conditions, including both continuous and discrete observations from single or multiple trajectories. To validate our theoretical results, we perform a series of simulations, which support and substantiate our findings.
SceneGenAgent: Precise Industrial Scene Generation with Coding Agent
Authors: Xiao Xia, Dan Zhang, Zibo Liao, Zhenyu Hou, Tianrui Sun, Jing Li, Ling Fu, Yuxiao Dong
Date: 2024-10-29 00:00:00
Tasks: C++ code, Scene Generation

The modeling of industrial scenes is essential for simulations in industrial manufacturing. While large language models (LLMs) have shown significant progress in generating general 3D scenes from textual descriptions, generating industrial scenes with LLMs poses a unique challenge due to their demand for precise measurements and positioning, requiring complex planning over spatial arrangement. To address this challenge, we introduce SceneGenAgent, an LLM-based agent for generating industrial scenes through C# code. SceneGenAgent ensures precise layout planning through a structured and calculable format, layout verification, and iterative refinement to meet the quantitative requirements of industrial scenarios. Experiment results demonstrate that LLMs powered by SceneGenAgent exceed their original performance, reaching up to 81.0% success rate in real-world industrial scene generation tasks and effectively meeting most scene generation requirements. To further enhance accessibility, we construct SceneInstruct, a dataset designed for fine-tuning open-source LLMs to integrate into SceneGenAgent. Experiments show that fine-tuning open-source LLMs on SceneInstruct yields significant performance improvements, with Llama3.1-70B approaching the capabilities of GPT-4o. Our code and data are available at https://github.com/THUDM/SceneGenAgent .
Evaluating K-Fold Cross Validation for Transformer Based Symbolic Regression Models
Authors: Kaustubh Kislay, Shlok Singh, Soham Joshi, Rohan Dutta, Jay Shim George Flint, Kevin Zhu
Date: 2024-10-29 00:00:00
Tasks: regression, Symbolic Regression

Symbolic Regression remains an NP-Hard problem, with extensive research focusing on AI models for this task. Transformer models have shown promise in Symbolic Regression, but performance suffers with smaller datasets. We propose applying k-fold cross-validation to a transformer-based symbolic regression model trained on a significantly reduced dataset (15,000 data points, down from 500,000). This technique partitions the training data into multiple subsets (folds), iteratively training on some while validating on others. Our aim is to provide an estimate of model generalization and mitigate overfitting issues associated with smaller datasets. Results show that this process improves the model's output consistency and generalization by a relative improvement in validation loss of 53.31%. Potentially enabling more efficient and accessible symbolic regression in resource-constrained environments.
SCGNet-Stacked Convolution with Gated Recurrent Unit Network for Cyber Network Intrusion Detection and Intrusion Type Classification
Authors: Rajana Akter, Shahnure Rabib, Rahul Deb Mohalder, Laboni Paul, Ferdous Bin Ali
Date: 2024-10-29 00:00:00
Tasks: Intrusion Detection, Network Intrusion Detection

Intrusion detection system (IDS) is a piece of hardware or software that looks for malicious activity or policy violations in a network. It looks for malicious activity or security flaws on a network or system. IDS protects hosts or networks by looking for indications of known attacks or deviations from normal behavior (Network-based intrusion detection system, or NIDS for short). Due to the rapidly increasing amount of network data, traditional intrusion detection systems (IDSs) are far from being able to quickly and efficiently identify complex and varied network attacks, especially those linked to low-frequency attacks. The SCGNet (Stacked Convolution with Gated Recurrent Unit Network) is a novel deep learning architecture that we propose in this study. It exhibits promising results on the NSL-KDD dataset in both task, network attack detection, and attack type classification with 99.76% and 98.92% accuracy, respectively. We have also introduced a general data preprocessing pipeline that is easily applicable to other similar datasets. We have also experimented with conventional machine-learning techniques to evaluate the performance of the data processing pipeline.
Hierarchical mixtures of Unigram models for short text clustering: The role of Beta-Liouville priors
Authors: Massimo Bilancia, Samuele Magro
Date: 2024-10-29 00:00:00
Tasks: Short Text Clustering, Text Clustering, Variational Inference

This paper presents a variant of the Multinomial mixture model tailored to the unsupervised classification of short text data. While the Multinomial probability vector is traditionally assigned a Dirichlet prior distribution, this work explores an alternative formulation based on the Beta-Liouville distribution, which offers a more flexible correlation structure than the Dirichlet. We examine the theoretical properties of the Beta-Liouville distribution, with particular focus on its conjugacy with the Multinomial likelihood. This property enables the derivation of update equations for a CAVI (Coordinate Ascent Variational Inference) algorithm, facilitating approximate posterior inference of the model parameters. In addition, we introduce a stochastic variant of the CAVI algorithm to enhance scalability. The paper concludes with empirical examples demonstrating effective strategies for selecting the Beta-Liouville hyperparameters.
Joint Estimation of Conditional Mean and Covariance for Unbalanced Panels
Authors: Damir Filipovic, Paul Schneider
Date: 2024-10-29 00:00:00

We develop a nonparametric, kernel-based joint estimator for conditional mean and covariance matrices in large and unbalanced panels. The estimator is supported by rigorous consistency results and finite-sample guarantees, ensuring its reliability for empirical applications. We apply it to an extensive panel of monthly US stock excess returns from 1962 to 2021, using macroeconomic and firm-specific covariates as conditioning variables. The estimator effectively captures time-varying cross-sectional dependencies, demonstrating robust statistical and economic performance. We find that idiosyncratic risk explains, on average, more than 75% of the cross-sectional variance.
Exponentially Consistent Statistical Classification of Continuous Sequences with Distribution Uncertainty
Authors: Lina Zhu, Lin Zhou
Date: 2024-10-29 00:00:00

In multiple classification, one aims to determine whether a testing sequence is generated from the same distribution as one of the M training sequences or not. Unlike most of existing studies that focus on discrete-valued sequences with perfect distribution match, we study multiple classification for continuous sequences with distribution uncertainty, where the generating distributions of the testing and training sequences deviate even under the true hypothesis. In particular, we propose distribution free tests and prove that the error probabilities of our tests decay exponentially fast for three different test designs: fixed-length, sequential, and two-phase tests. We first consider the simple case without the null hypothesis, where the testing sequence is known to be generated from a distribution close to the generating distribution of one of the training sequences. Subsequently, we generalize our results to a more general case with the null hypothesis by allowing the testing sequence to be generated from a distribution that is vastly different from the generating distributions of all training sequences.
Reliable and Compact Graph Fine-tuning via GraphSparse Prompting
Authors: Bo Jiang, Hao Wu, Beibei Wang, Jin Tang, Bin Luo
Date: 2024-10-29 00:00:00
Tasks: Attribute, Graph Learning, Prompt Learning

Recently, graph prompt learning has garnered increasing attention in adapting pre-trained GNN models for downstream graph learning tasks. However, existing works generally conduct prompting over all graph elements (e.g., nodes, edges, node attributes, etc.), which is suboptimal and obviously redundant. To address this issue, we propose exploiting sparse representation theory for graph prompting and present Graph Sparse Prompting (GSP). GSP aims to adaptively and sparsely select the optimal elements (e.g., certain node attributes) to achieve compact prompting for downstream tasks. Specifically, we propose two kinds of GSP models, termed Graph Sparse Feature Prompting (GSFP) and Graph Sparse multi-Feature Prompting (GSmFP). Both GSFP and GSmFP provide a general scheme for tuning any specific pre-trained GNNs that can achieve attribute selection and compact prompt learning simultaneously. A simple yet effective algorithm has been designed for solving GSFP and GSmFP models. Experiments on 16 widely-used benchmark datasets validate the effectiveness and advantages of the proposed GSFPs.
RDSA: A Robust Deep Graph Clustering Framework via Dual Soft Assignment
Authors: Yang Xiang, Li Fan, Tulika Saha, Xiaoying Pang, Yushan Pan, Haiyang Zhang, Chengtao Ji
Date: 2024-10-29 00:00:00
Tasks: Clustering, Denoising, Graph Clustering

Graph clustering is an essential aspect of network analysis that involves grouping nodes into separate clusters. Recent developments in deep learning have resulted in graph clustering, which has proven effective in many applications. Nonetheless, these methods often encounter difficulties when dealing with real-world graphs, particularly in the presence of noisy edges. Additionally, many denoising graph clustering methods tend to suffer from lower performance, training instability, and challenges in scaling to large datasets compared to non-denoised models. To tackle these issues, we introduce a new framework called the Robust Deep Graph Clustering Framework via Dual Soft Assignment (RDSA). RDSA consists of three key components: (i) a node embedding module that effectively integrates the graph's topological features and node attributes; (ii) a structure-based soft assignment module that improves graph modularity by utilizing an affinity matrix for node assignments; and (iii) a node-based soft assignment module that identifies community landmarks and refines node assignments to enhance the model's robustness. We assess RDSA on various real-world datasets, demonstrating its superior performance relative to existing state-of-the-art methods. Our findings indicate that RDSA provides robust clustering across different graph types, excelling in clustering effectiveness and robustness, including adaptability to noise, stability, and scalability.
Sliced-Wasserstein-based Anomaly Detection and Open Dataset for Localized Critical Peak Rebates
Authors: Julien Pallage, Bertrand Scherrer, Salma Naccache, Christophe Bélanger, Antoine Lesage-Landry
Date: 2024-10-29 00:00:00
Tasks: Anomaly Detection, Outlier Detection

In this work, we present a new unsupervised anomaly (outlier) detection (AD) method using the sliced-Wasserstein metric. This filtering technique is conceptually interesting for MLOps pipelines deploying machine learning models in critical sectors, e.g., energy, as it offers a conservative data selection. Additionally, we open the first dataset showcasing localized critical peak rebate demand response in a northern climate. We demonstrate the capabilities of our method on synthetic datasets as well as standard AD datasets and use it in the making of a first benchmark for our open-source localized critical peak rebate dataset.
Multi-view clustering integrating anchor attribute and structural information
Authors: Xuetong Li, Xiao-Dong Zhang
Date: 2024-10-29 00:00:00
Tasks: Attribute, Clustering

Multisource data has spurred the development of advanced clustering algorithms, such as multi-view clustering, which critically relies on constructing similarity matrices. Traditional algorithms typically generate these matrices from sample attributes alone. However, real-world networks often include pairwise directed topological structures critical for clustering. This paper introduces a novel multi-view clustering algorithm, AAS. It utilizes a two-step proximity approach via anchors in each view, integrating attribute and directed structural information. This approach enhances the clarity of category characteristics in the similarity matrices. The anchor structural similarity matrix leverages strongly connected components of directed graphs. The entire process-from similarity matrices construction to clustering - is consolidated into a unified optimization framework. Comparative experiments on the modified Attribute SBM dataset against eight algorithms affirm the effectiveness and superiority of AAS.
Stochastic Approximation with Unbounded Markovian Noise: A General-Purpose Theorem
Authors: Shaan ul Haque, Siva Theja Maguluri
Date: 2024-10-29 00:00:00
Tasks: Q-Learning, Stochastic Optimization

Motivated by engineering applications such as resource allocation in networks and inventory systems, we consider average-reward Reinforcement Learning with unbounded state space and reward function. Recent works studied this problem in the actor-critic framework and established finite sample bounds assuming access to a critic with certain error guarantees. We complement their work by studying Temporal Difference (TD) learning with linear function approximation and establishing finite-time bounds with the optimal $\mathcal{O}\left(1/\epsilon^2\right)$ sample complexity. These results are obtained using the following general-purpose theorem for non-linear Stochastic Approximation (SA). Suppose that one constructs a Lyapunov function for a non-linear SA with certain drift condition. Then, our theorem establishes finite-time bounds when this SA is driven by unbounded Markovian noise under suitable conditions. It serves as a black box tool to generalize sample guarantees on SA from i.i.d. or martingale difference case to potentially unbounded Markovian noise. The generality and the mild assumption of the setup enables broad applicability of our theorem. We illustrate its power by studying two more systems: (i) We improve upon the finite-time bounds of $Q$-learning by tightening the error bounds and also allowing for a larger class of behavior policies. (ii) We establish the first ever finite-time bounds for distributed stochastic optimization of high-dimensional smooth strongly convex function using cyclic block coordinate descent.