OpenRaiser/Intern-Atlas huggingface dataset:
On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients
Venue: arXiv cs.LG 2025
Year: 2025

The holy grail of machine learning is to enable Continual Federated Learning (CFL) to enhance the efficiency, privacy, and scalability of AI systems while learning from streaming data. The primary challenge of a CFL system is to overcome global catastrophic forgetting, wherein the accuracy of the global model trained on new tasks declines on the old tasks. In this work, we propose Continual Federated Learning with Aggregated Gradients (C-FLAG), a novel replay-memory based federated strategy consisting of edge-based gradient updates on memory and aggregated gradients on the current data. We provide convergence analysis of the C-FLAG approach which addresses forgetting and bias while converging at a rate of $O(1/\sqrt{T})$ over $T$ communication rounds. We formulate an optimization sub-problem that minimizes catastrophic forgetting, translating CFL into an iterative algorithm with adaptive learning rates that ensure seamless learning across tasks. We empirically show that C-FLAG outperforms several state-of-the-art baselines on both task and class-incremental settings with respect to metrics such as accuracy and forgetting.
Impactful Bit-Flip Search on Full-precision Models
Venue: arXiv cs.LG 2025
Year: 2025

Neural networks have shown remarkable performance in various tasks, yet they remain susceptible to subtle changes in their input or model parameters. One particularly impactful vulnerability arises through the Bit-Flip Attack (BFA), where flipping a small number of critical bits in a model's parameters can severely degrade its performance. A common technique for inducing bit flips in DRAM is the Row-Hammer attack, which exploits frequent uncached memory accesses to alter data. Identifying susceptible bits can be achieved through exhaustive search or progressive layer-by-layer analysis, especially in quantized networks. In this work, we introduce Impactful Bit-Flip Search (IBS), a novel method for efficiently pinpointing and flipping critical bits in full-precision networks. Additionally, we propose a Weight-Stealth technique that strategically modifies the model's parameters in a way that maintains the float values within the original distribution, thereby bypassing simple range checks often used in tamper detection.
Conditional Variable Flow Matching: Transforming Conditional Densities with Amortized Conditional Optimal Transport
Venue: arXiv cs.LG 2025
Year: 2025

Forecasting conditional stochastic nonlinear dynamical systems is a fundamental challenge repeatedly encountered across the biological and physical sciences. While flow-based models can impressively predict the temporal evolution of probability distributions representing possible outcomes of a specific process, existing frameworks cannot satisfactorily account for the impact of conditioning variables on these dynamics. Amongst several limitations, existing methods require training data with paired conditions and are developed for discrete conditioning variables. We propose Conditional Variable Flow Matching (CVFM), a framework for learning flows transforming conditional distributions with amortization across continuous conditioning variables - permitting predictions across the conditional density manifold. This is accomplished through several novel advances. In particular, simultaneous sample conditioned flows over the main and conditioning variables, alongside a conditional Wasserstein distance combined with a loss reweighting kernel facilitating conditional optimal transport. Collectively, these advances allow for learning system dynamics provided measurement data whose states and conditioning variables are not in correspondence. We demonstrate CVFM on a suite of increasingly challenging problems, including discrete and continuous conditional mapping benchmarks, image-to-image domain transfer, and modeling the temporal evolution of materials internal structure during manufacturing processes. We observe that CVFM results in improved performance and convergence characteristics over alternative conditional variants.
Anonymous Self-Stabilising Localisation via Spatial Population Protocols
Venue: arXiv cs.DC 2025
Year: 2025

In the distributed localization problem (DLP), $n$ anonymous robots (agents) $a_0, a_1, ..., a_{n-1}$ begin at arbitrary positions $p_0, ..., p_{n-1}$ in $S$, where $S$ is an Euclidean space. The primary goal in DLP is for agents to reach a consensus on a unified coordinate system that accurately reflects the relative positions of all points, $p_0, ..., p_{n-1}$. Extensive research on DLP has primarily focused on the feasibility and complexity of achieving consensus when agents have limited access to inter-agent distances, often due to missing or imprecise data. In this paper, however, we examine a minimalist, computationally efficient model of distributed computing in which agents have access to all pairwise distances, if needed. Specifically, we introduce a novel variant of population protocols, referred to as the spatial population protocols model. In this variant each agent can memorise one or a fixed number of coordinates, and when agents $a_i$ and $a_j$ interact, they can not only exchange their current knowledge but also either determine the distance $d(i,j)$ between them in $S$ (distance query model) or obtain the vector $v(i,j)$ spanning points $p_i$ and $p_j$ (vector query model). We propose several localisation protocols, including: (1) Two leader-based protocols with distance queries, stabilizing silently in $o(n)$ time using an efficient multi-contact epidemic, a generalization of the one-way epidemic in population protocols; (2) A distance-based protocol self-stabilizing silently in $O(n(\log n/n)^{1/(k+1)}\log n)$ time in $k$-dimensions, leveraging a leader election mechanism; (3) An optimally fast protocol with vector queries, self-stabilizing silently in $O(\log n)$ time.
Achieving Consistent and Comparable CPU Evaluation
Venue: arXiv cs.PF 2025
Year: 2025

The challenge of CPU evaluation lies in the fact that user-perceived performance metrics can only be measured on an independently running system consisting of the CPU and other indispensable components, and hence it is difficult to accurately attribute the deviations in the evaluation outcomes to the differences between the CPUs. Our experiments reveal that the industry-standard CPU benchmark, SPEC CPU2017, suffers from a significant flaw: for the identical CPU, undefined configurations of other indispensable components introduce uncontrolled variability in evaluation outcomes. We propose a rigorous CPU evaluation methodology. Through theoretical analysis and pioneering controlled experiments, we systematically compare our methodology against four established methodologies: the SPEC CPU 2017, two DOE variants, and one RCTs approach. The results show our methodology can achieve consistent and comparable evaluation outcomes, while others exhibit inherent limations.
Neural Corrective Machine Unranking
Venue: arXiv cs.IR 2025
Year: 2025

Machine unlearning in neural information retrieval (IR) systems requires removing specific data whilst maintaining model performance. Applying existing machine unlearning methods to IR may compromise retrieval effectiveness or inadvertently expose unlearning actions due to the removal of particular items from the retrieved results presented to users. We formalise corrective unranking, which extends machine unlearning in (neural) IR context by integrating substitute documents to preserve ranking integrity, and propose a novel teacher-student framework, Corrective unRanking Distillation (CuRD), for this task. CuRD (1) facilitates forgetting by adjusting the (trained) neural IR model such that its output relevance scores of to-be-forgotten samples mimic those of low-ranking, non-retrievable samples; (2) enables correction by fine-tuning the relevance scores for the substitute samples to match those of corresponding to-be-forgotten samples closely; (3) seeks to preserve performance on samples that are not targeted for forgetting. We evaluate CuRD on four neural IR models (BERTcat, BERTdot, ColBERT, PARADE) using MS MARCO and TREC CAR datasets. Experiments with forget set sizes from 1 % and 20 % of the training dataset demonstrate that CuRD outperforms seven state-of-the-art baselines in terms of forgetting and correction while maintaining model retention and generalisation capabilities.
Themes of Building LLM-based Applications for Production: A Practitioner's View
Venue: arXiv cs.SE 2025
Year: 2025

Background: Large language models (LLMs) have become a paramount interest of researchers and practitioners alike, yet a comprehensive overview of key considerations for those developing LLM-based systems is lacking. This study addresses this gap by collecting and mapping the topics practitioners discuss online, offering practical insights into where priorities lie in developing LLM-based applications. Method: We collected 189 videos from 2022 to 2024 from practitioners actively developing such systems and discussing various aspects they encounter during development and deployment of LLMs in production. We analyzed the transcripts using BERTopic, then manually sorted and merged the generated topics into themes, leading to a total of 20 topics in 8 themes. Results: The most prevalent topics fall within the theme Design & Architecture, with a strong focus on retrieval-augmented generation (RAG) systems. Other frequently discussed topics include model capabilities and enhancement techniques (e.g., fine-tuning, prompt engineering), infrastructure and tooling, and risks and ethical challenges. Implications: Our results highlight current discussions and challenges in deploying LLMs in production. This way, we provide a systematic overview of key aspects practitioners should be aware of when developing LLM-based applications. We further pale off topics of interest for academics where further research is needed.
A Preview of XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL
Venue: arXiv cs.AI 2025
Year: 2025

To tackle the challenges of large language model performance in natural language to SQL tasks, we introduce XiYan-SQL, an innovative framework that employs a multi-generator ensemble strategy to improve candidate generation. We introduce M-Schema, a semi-structured schema representation method designed to enhance the understanding of database structures. To enhance the quality and diversity of generated candidate SQL queries, XiYan-SQL integrates the significant potential of in-context learning (ICL) with the precise control of supervised fine-tuning. On one hand, we propose a series of training strategies to fine-tune models to generate high-quality candidates with diverse preferences. On the other hand, we implement the ICL approach with an example selection method based on named entity recognition to prevent overemphasis on entities. The refiner optimizes each candidate by correcting logical or syntactical errors. To address the challenge of identifying the best candidate, we fine-tune a selection model to distinguish nuances of candidate SQL queries. The experimental results on multiple dialect datasets demonstrate the robustness of XiYan-SQL in addressing challenges across different scenarios. Overall, our proposed XiYan-SQL achieves the state-of-the-art execution accuracy of 75.63% on Bird benchmark, 89.65% on the Spider test set, 69.86% on SQL-Eval, 41.20% on NL2GQL. The proposed framework not only enhances the quality and diversity of SQL queries but also outperforms previous methods.
Learning-Guided Fuzzing for Testing Stateful SDN Controllers
Venue: arXiv cs.SE 2025
Year: 2025

Controllers for software-defined networks (SDNs) are centralised software components that enable advanced network functionalities, such as dynamic traffic engineering and network virtualisation. However, these functionalities increase the complexity of SDN controllers, making thorough testing crucial. SDN controllers are stateful, interacting with multiple network devices through sequences of control messages. Identifying stateful failures in an SDN controller is challenging due to the infinite possible sequences of control messages, which result in an unbounded number of stateful interactions between the controller and network devices. In this article, we propose SeqFuzzSDN, a learning-guided fuzzing method for testing stateful SDN controllers. SeqFuzzSDN aims to (1) efficiently explore the state space of the SDN controller under test, (2) generate effective and diverse tests (i.e., control message sequences) to uncover failures, and (3) infer accurate failure-inducing models that characterise the message sequences leading to failures. In addition, we compare SeqFuzzSDN with three extensions of state-of-the-art (SOTA) methods for fuzzing SDNs. Our findings show that, compared to the extended SOTA methods, SeqFuzzSDN (1) generates more diverse message sequences that lead to failures within the same time budget, and (2) produces more accurate failure-inducing models, significantly outperforming the other extended SOTA methods in terms of sensitivity.
Bayesian Comparisons Between Representations
Venue: arXiv cs.LG 2025
Year: 2025

Which neural networks are similar is a fundamental question for both machine learning and neuroscience. Here, it is proposed to base comparisons on the predictive distributions of linear readouts from intermediate representations. In Bayesian statistics, the prior predictive distribution is a full description of the inductive bias and generalization of a model, making it a great basis for comparisons. This distribution directly gives the evidence a dataset would provide in favor of the model. If we want to compare multiple models to each other, we can use a metric for probability distributions like the Jensen-Shannon distance or the total variation distance. As these are metrics, this induces pseudo-metrics for representations, which measure how well two representations could be distinguished based on a linear read out. For a linear readout with a Gaussian prior on the read-out weights and Gaussian noise, we can analytically compute the (prior and posterior) predictive distributions without approximations. These distributions depend only on the linear kernel matrix of the representations in the model. Thus, the Bayesian metrics connect to both linear read-out based comparisons and kernel based metrics like centered kernel alignment and representational similarity analysis. The new methods are demonstrated with deep neural networks trained on ImageNet-1k comparing them to each other and a small subset of the Natural Scenes Dataset. The Bayesian comparisons are correlated to but distinct from existing metrics. Evaluations vary slightly less across random image samples and yield informative results with full uncertainty information. Thus the proposed Bayesian metrics nicely extend our toolkit for comparing representations.
The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models
Venue: arXiv cs.CL 2025
Year: 2025

Several recent works seek to adapt general-purpose large language models (LLMs) and vision-language models (VLMs) for medical applications through continued pretraining on publicly available biomedical corpora. These works typically claim that such domain-adaptive pretraining improves performance on various downstream medical tasks, such as answering medical exam questions. In this paper, we compare ten "medical" LLMs and two VLMs against their corresponding base models, arriving at a different conclusion: all medical VLMs and nearly all medical LLMs fail to consistently improve over their base models in the zero-/few-shot prompting and supervised fine-tuning regimes for medical question answering (QA). For instance, on clinical-note-based QA tasks in the 3-shot setting, medical LLMs outperform their base models in only 26.7% of cases, reach a (statistical) tie in 16.7% of cases, and perform significantly worse in the remaining 56.7% of cases. Our conclusions are based on (i) comparing each medical model directly against its base model; (ii) optimizing the prompts for each model separately in zero-/few-shot prompting; and (iii) accounting for statistical uncertainty in comparisons. Our findings suggest that state-of-the-art general-domain models may already exhibit strong medical knowledge and reasoning capabilities, and offer recommendations to strengthen the conclusions of future studies.
Quantifying Risk Propensities of Large Language Models: Ethical Focus and Bias Detection through Role-Play
Venue: arXiv cs.CY 2025
Year: 2025

As Large Language Models (LLMs) become more prevalent, concerns about their safety, ethics, and potential biases have risen. Systematically evaluating LLMs' risk decision-making tendencies and attitudes, particularly in the ethical domain, has become crucial. This study innovatively applies the Domain-Specific Risk-Taking (DOSPERT) scale from cognitive science to LLMs and proposes a novel Ethical Decision-Making Risk Attitude Scale (EDRAS) to assess LLMs' ethical risk attitudes in depth. We further propose a novel approach integrating risk scales and role-playing to quantitatively evaluate systematic biases in LLMs. Through systematic evaluation and analysis of multiple mainstream LLMs, we assessed the "risk personalities" of LLMs across multiple domains, with a particular focus on the ethical domain, and revealed and quantified LLMs' systematic biases towards different groups. This research helps understand LLMs' risk decision-making and ensure their safe and reliable application. Our approach provides a tool for identifying and mitigating biases, contributing to fairer and more trustworthy AI systems. The code and data are available.
Performance-Complexity-Latency Trade-offs of Concatenated RS-SDBCH Codes
Venue: arXiv cs.IT 2025
Year: 2025

Concatenated bit-interleaved and multilevel coded modulation with outer Reed--Solomon codes, inner Chase-algorithm-based soft-decision-decoded Bose--Ray-Chaudhuri--Hocquenghem codes, and four-level pulse amplitude modulation is considered. A semi-analytical formula is derived for estimating the decoded frame error rate (FER) at the output of the additive white Gaussian noise channel, obviating the need for time-consuming Monte Carlo simulations. The formula is used to search a large space of codes (including the KP4 code) to find those achieving good trade-offs among performance (measured by the gap to the constrained Shannon limit at $10^{-13}$ FER), complexity (measured by the number of elementary decoder operations), and latency (measured by overall block length).
The Geometry of Codes for Random Access in DNA Storage
Venue: arXiv cs.IT 2025
Year: 2025

Effective and reliable data retrieval is critical for the feasibility of DNA storage, and the development of random access efficiency plays a key role in its practicality and reliability. In this paper, we study the Random Access Problem, which asks to compute the expected number of samples one needs in order to recover an information strand. Unlike previous work, we took a geometric approach to the problem, aiming to understand which geometric structures lead to codes that perform well in terms of reducing the random access expectation (Balanced Quasi-Arcs). As a consequence, two main results are obtained. The first is a construction for $k=3$ that outperforms previous constructions aiming to reduce the random access expectation. The second, exploiting a result from~\cite{gruica2024reducing}, is the proof of a conjecture from~\cite{bar2023cover} for rate $1/2$ codes in any dimension.
Flexible Modeling of Information Diffusion on Networks with Statistical Guarantees
Venue: arXiv cs.SI 2025
Year: 2025

Modeling information spread through a network is one of the key problems of network analysis, with applications in a wide array of areas such as marketing and public health. Most approaches assume that the spread is governed by some probabilistic diffusion model, often parameterized by the strength of connections between network members (edge weights), highlighting the need for methods that can accurately estimate them. Multiple prior works suggest such estimators for particular diffusion models; however, most of them lack a rigorous statistical analysis that would establish the asymptotic properties of the estimator and allow for uncertainty quantification. In this paper, we develop a likelihood-based approach to estimate edge weights from the observed information diffusion paths under the proposed General Linear Threshold (GLT) model, a broad class of discrete-time information diffusion models that includes both the well-known linear threshold (LT) and independent cascade (IC) models. We first derive necessary and sufficient conditions that make the edge weights identifiable under this model. Then, we derive a finite sample error bound for the estimator and demonstrate that it is asymptotically normal under mild conditions. We conclude by studying the GLT model in the context of the Influence Maximization (IM) problem, that is, the task of selecting a subset of $k$ nodes to start the diffusion, so that the average information spread is maximized. Extensive experiments on synthetic and real-world networks demonstrate that the flexibility of the proposed class of GLT models, coupled with the proposed estimation and inference framework for its parameters, can significantly improve estimation of spread from a given subset of nodes, prediction of node activation, and the quality of the IM problem solutions.
SAFES: Sequential Privacy and Fairness Enhancing Data Synthesis for Responsible AI
Venue: arXiv cs.LG 2025
Year: 2025

As data-driven and AI-based decision making gains widespread adoption across disciplines, it is crucial that both data privacy and decision fairness are appropriately addressed. Although differential privacy (DP) provides a robust framework for guaranteeing privacy and methods are available to improve fairness, most prior work treats the two concerns separately. Even though there are existing approaches that consider privacy and fairness simultaneously, they typically focus on a single specific learning task, limiting their generalizability. In response, we introduce SAFES, a Sequential PrivAcy and Fairness Enhancing data Synthesis procedure that sequentially combines DP data synthesis with a fairness-aware data preprocessing step. SAFES allows users flexibility in navigating the privacy-fairness-utility trade-offs. We illustrate SAFES with different DP synthesizers and fairness-aware data preprocessing methods and run extensive experiments on multiple real datasets to examine the privacy-fairness-utility trade-offs of synthetic data generated by SAFES. Empirical evaluations demonstrate that for reasonable privacy loss, SAFES-generated synthetic data can achieve significantly improved fairness metrics with relatively low utility loss.