OpenRaiser/Intern-Atlas huggingface dataset:
In-Distribution Consistency Regularization Improves the Generalization of Quantization-Aware Training
Venue: arXiv cs.CV 2025
Year: 2025

Although existing Quantization-Aware Training (QAT) methods intensively depend on knowledge distillation to guarantee performance, QAT still suffers from severe performance drop. The experiments have shown that vanilla quantization is sensitive to the perturbation from both the input and weights. Therefore, we assume that the generalization ability of QAT is predominantly caused by both the intrinsic instability (training time) and the limited generalization ability (testing time). In this paper, we address both issues from a new perspective by leveraging Consistency Regularization (CR) to improve the generalization ability of QAT. Empirical results and theoretical analysis verify that CR would bring a good generalization ability to different network architectures and various QAT methods. Extensive experiments demonstrate that our approach significantly outperforms current state-of-the-art QAT methods and even the FP counterparts. On CIFAR-10, the proposed method improves by 3.79% compared to the baseline method using ResNet18, and improves by 3.84% compared to the baseline method using the lightweight model MobileNet.
Transforming Norm-based To Graph-based Spatial Representation for Spatio-Temporal Epidemiological Models
Venue: arXiv cs.IR 2025
Year: 2025

Pandemics, with their profound societal and economic impacts, pose significant threats to global health, mortality rates, economic stability, and political landscapes. In response to these challenges, numerous studies have employed spatio-temporal models to enhance our understanding and management of these complex phenomena. These spatio-temporal models can be roughly divided into two main spatial categories: norm-based and graph-based. Norm-based models are usually more accurate and easier to model but are more computationally intensive and require more data to fit. On the other hand, graph-based models are less accurate and harder to model but are less computationally intensive and require fewer data to fit. As such, ideally, one would like to use a graph-based model while preserving the representation accuracy obtained by the norm-based model. In this study, we explore the ability to transform from norm-based to graph-based spatial representation for these models. We first show no analytical mapping between the two exists, requiring one to use approximation numerical methods instead. We introduce a novel framework for this task together with twelve possible implementations using a wide range of heuristic optimization approaches. Our findings show that by leveraging agent-based simulations and heuristic algorithms for the graph node's location and population's spatial walk dynamics approximation one can use graph-based spatial representation without losing much of the model's accuracy and expressiveness. We investigate our framework for three real-world cases, achieving 94\% accuracy preservation, on average. Moreover, an analysis of synthetic cases shows the proposed framework is relatively robust for changes in both spatial and temporal properties.
EMN: Brain-inspired Elastic Memory Network for Quick Domain Adaptive Feature Mapping
Venue: arXiv cs.NE 2025
Year: 2025

Utilizing unlabeled data in the target domain to perform continuous optimization is critical to enhance the generalization ability of neural networks. Most domain adaptation methods focus on time-consuming optimization of deep feature extractors, which limits the deployment on lightweight edge devices. Inspired by the memory mechanism and powerful generalization ability of biological neural networks in human brains, we propose a novel gradient-free Elastic Memory Network, namely EMN, to support quick fine-tuning of the mapping between features and prediction without heavy optimization of deep features. In particular, EMN adopts randomly connected neurons to memorize the association of features and labels, where the signals in the network are propagated as impulses, and the prediction is made by associating the memories stored on neurons based on their confidence. More importantly, EMN supports reinforced memorization of feature mapping based on unlabeled data to quickly adapt to a new domain. Experiments based on four cross-domain real-world datasets show that EMN can achieve up to 10% enhancement of performance while only needing less than 1% timing cost of traditional domain adaptation methods.
OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
Venue: arXiv cs.SE 2025
Year: 2025

The introduction of large language models has significantly advanced code generation. However, open-source models often lack the execution capabilities and iterative refinement of advanced systems like the GPT-4 Code Interpreter. To address this, we introduce OpenCodeInterpreter, a family of open-source code systems designed for generating, executing, and iteratively refining code. Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions, OpenCodeInterpreter integrates execution and human feedback for dynamic code refinement. Our comprehensive evaluation of OpenCodeInterpreter across key benchmarks such as HumanEval, MBPP, and their enhanced versions from EvalPlus reveals its exceptional performance. Notably, OpenCodeInterpreter-33B achieves an accuracy of 83.2 (76.4) on the average (and plus versions) of HumanEval and MBPP, closely rivaling GPT-4's 84.2 (76.2) and further elevates to 91.6 (84.6) with synthesized human feedback from GPT-4. OpenCodeInterpreter brings the gap between open-source code generation models and proprietary systems like GPT-4 Code Interpreter.
Bayesian Off-Policy Evaluation and Learning for Large Action Spaces
Venue: arXiv cs.LG 2025
Year: 2025

In interactive systems, actions are often correlated, presenting an opportunity for more sample-efficient off-policy evaluation (OPE) and learning (OPL) in large action spaces. We introduce a unified Bayesian framework to capture these correlations through structured and informative priors. In this framework, we propose sDM, a generic Bayesian approach for OPE and OPL, grounded in both algorithmic and theoretical foundations. Notably, sDM leverages action correlations without compromising computational efficiency. Moreover, inspired by online Bayesian bandits, we introduce Bayesian metrics that assess the average performance of algorithms across multiple problem instances, deviating from the conventional worst-case assessments. We analyze sDM in OPE and OPL, highlighting the benefits of leveraging action correlations. Empirical evidence showcases the strong performance of sDM.
Convergence Analysis of Split Federated Learning on Heterogeneous Data
Venue: arXiv cs.DC 2025
Year: 2025

Split federated learning (SFL) is a recent distributed approach for collaborative model training among multiple clients. In SFL, a global model is typically split into two parts, where clients train one part in a parallel federated manner, and a main server trains the other. Despite the recent research on SFL algorithm development, the convergence analysis of SFL is missing in the literature, and this paper aims to fill this gap. The analysis of SFL can be more challenging than that of federated learning (FL), due to the potential dual-paced updates at the clients and the main server. We provide convergence analysis of SFL for strongly convex and general convex objectives on heterogeneous data. The convergence rates are $O(1/T)$ and $O(1/\sqrt[3]{T})$, respectively, where $T$ denotes the total number of rounds for SFL training. We further extend the analysis to non-convex objectives and the scenario where some clients may be unavailable during training. Experimental experiments validate our theoretical results and show that SFL outperforms FL and split learning (SL) when data is highly heterogeneous across a large number of clients.
Label-efficient multi-organ segmentation with a diffusion model
Venue: arXiv cs.CV 2025
Year: 2025

Accurate segmentation of multiple organs in Computed Tomography (CT) images plays a vital role in computer-aided diagnosis systems. While various supervised learning approaches have been proposed recently, these methods heavily depend on a large amount of high-quality labeled data, which are expensive to obtain in practice. To address this challenge, we propose a label-efficient framework using knowledge transfer from a pre-trained diffusion model for CT multi-organ segmentation. Specifically, we first pre-train a denoising diffusion model on 207,029 unlabeled 2D CT slices to capture anatomical patterns. Then, the model backbone is transferred to the downstream multi-organ segmentation task, followed by fine-tuning with few labeled data. In fine-tuning, two fine-tuning strategies, linear classification and fine-tuning decoder, are employed to enhance segmentation performance while preserving learned representations. Quantitative results show that the pre-trained diffusion model is capable of generating diverse and realistic 256x256 CT images (Fréchet inception distance (FID): 11.32, spatial Fréchet inception distance (sFID): 46.93, F1-score: 73.1%). Compared to state-of-the-art methods for multi-organ segmentation, our method achieves competitive performance on the FLARE 2022 dataset, particularly in limited labeled data scenarios. After fine-tuning with 1% and 10% labeled data, our method achieves dice similarity coefficients (DSCs) of 71.56% and 78.51%, respectively. Remarkably, the method achieves a DSC score of 51.81% using only four labeled CT slices. These results demonstrate the efficacy of our approach in overcoming the limitations of supervised learning approaches that is highly dependent on large-scale labeled data.
Open Energy Services -- Forecasting and Optimization as a Service for Energy Management Applications at Scale
Venue: arXiv cs.SE 2025
Year: 2025

This article aims at facilitating the widespread application of Energy Management Systems (EMSs), especially on buildings and cities, in order to support the realization of future carbon-neutral energy systems. We claim that economic viability is a severe issue for the utilization of EMSs at scale and that the provisioning of forecasting and optimization algorithms as a service can make a major contribution to achieve it. To this end, we present the \emph{Energy Service Generics} software framework that allows the derivation of fully functional services from existing forecasting or optimization code with ease. This work documents the strictly systematic development of the framework, beginning with a requirement analysis, from which a sophisticated design concept is derived, followed by a description of the implementation of the framework. Furthermore, we present the concept of the \emph{Open Energy Service} community, our effort to continuously maintain the service framework but also provide ready-to-use forecasting and optimization services. Finally, an evaluation of our framework and community concept, as well as a demarcation between our work and the current state of the art, is presented.
GPTVQ: The Blessing of Dimensionality for LLM Quantization
Venue: arXiv cs.LG 2025
Year: 2025

In this work we show that the size versus accuracy trade-off of neural network quantization can be significantly improved by increasing the quantization dimensionality. We propose the GPTVQ method, a new fast method for post-training vector quantization (VQ) that scales well to Large Language Models (LLMs). Our method interleaves quantization of one or more columns with updates to the remaining unquantized weights, using information from the Hessian of the per-layer output reconstruction MSE. Quantization codebooks are initialized using an efficient data-aware version of the EM algorithm. The codebooks are then updated, and further compressed by using integer quantization and SVD-based compression. GPTVQ establishes a new state-of-the art in the size vs accuracy trade-offs on a wide range of LLMs such as Llama-v2 and Mistral. Furthermore, our method is efficient: on a single H100 it takes between 3 and 11 hours to process a Llamav2-70B model, depending on quantization setting. Lastly, with on-device timings for VQ decompression on a mobile CPU we show that VQ leads to improved latency compared to using a 4-bit integer format.
Symmetric Rank-$1$ Regularization for Iterative Inversion of Ill-Conditioned MIMO Channels
Venue: arXiv cs.IT 2025
Year: 2025

While iterative matrix inversion methods excel in computational efficiency, memory optimization, and support for parallel and distributed computing when managing large matrices, their limitations are also evident in multiple-input multiple-output (MIMO) fading channels. These methods encounter challenges related to slow convergence and diminished accuracy, especially in ill-conditioned scenarios, hindering their application in future MIMO networks such as extra-large aperture array. To address these challenges, this paper proposes a novel matrix regularization method termed symmetric rank-$1$ regularization (SR-$1$R). The proposed method functions by augmenting the channel matrix with a symmetric rank-$1$ matrix, with the primary goal of minimizing the condition number of the resultant regularized matrix. This significantly improves the matrix condition, enabling fast and accurate iterative inversion of the regularized matrix. Then, the inverse of the original channel matrix is obtained by applying the Sherman-Morrison transform on the outcome of iterative inversions. Our eigenvalue analysis unveils the best channel condition that can be achieved by an optimized SR-$1$R matrix. Moreover, a power iteration-assisted (PIA) approach is proposed to find the optimum SR-$1$R matrix without need of eigenvalue decomposition. The proposed approach exhibits logarithmic algorithm-depth in parallel computing for MIMO precoding. Finally, computer simulations demonstrate that SR-$1$R has the potential to reduce the required iteration by up to $35\%$ while achieving the performance of regularized zero-forcing.
BlockFUL: Enabling Unlearning in Blockchained Federated Learning
Venue: arXiv cs.CR 2025
Year: 2025

Unlearning in Federated Learning (FL) presents significant challenges, as models grow and evolve with complex inheritance relationships. This complexity is amplified when blockchain is employed to ensure the integrity and traceability of FL, where the need to edit multiple interlinked blockchain records and update all inherited models complicates the process.In this paper, we introduce Blockchained Federated Unlearning (BlockFUL), a novel framework with a dual-chain structure comprising a live chain and an archive chain for enabling unlearning capabilities within Blockchained FL. BlockFUL introduces two new unlearning paradigms, i.e., parallel and sequential paradigms, which can be effectively implemented through gradient-ascent-based and re-training-based unlearning methods. These methods enhance the unlearning process across multiple inherited models by enabling efficient consensus operations and reducing computational costs. Our extensive experiments validate that these methods effectively reduce data dependency and operational overhead, thereby boosting the overall performance of unlearning inherited models within BlockFUL on CIFAR-10 and Fashion-MNIST datasets using AlexNet, ResNet18, and MobileNetV2 models.
Achieving Instance-dependent Sample Complexity for Constrained Markov Decision Process
Venue: arXiv cs.LG 2025
Year: 2025

We consider the reinforcement learning problem for the constrained Markov decision process (CMDP), which plays a central role in satisfying safety or resource constraints in sequential learning and decision-making. In this problem, we are given finite resources and a MDP with unknown transition probabilities. At each stage, we take an action, collecting a reward and consuming some resources, all assumed to be unknown and need to be learned over time. In this work, we take the first step towards deriving optimal problem-dependent guarantees for the CMDP problems. We derive a logarithmic regret bound, which translates into a $O(\frac{1}{Δ\cdotε}\cdot\log^2(1/ε))$ sample complexity bound, with $Δ$ being a problem-dependent parameter, yet independent of $ε$. Our sample complexity bound improves upon the state-of-art $O(1/ε^2)$ sample complexity for CMDP problems established in the previous literature, in terms of the dependency on $ε$. To achieve this advance, we develop a new framework for analyzing CMDP problems. To be specific, our algorithm operates in the primal space and we resolve the primal LP for the CMDP problem at each period in an online manner, with adaptive remaining resource capacities. The key elements of our algorithm are: i) a characterization of the instance hardness via LP basis, ii) an eliminating procedure that identifies one optimal basis of the primal LP, and; iii) a resolving procedure that is adaptive to the remaining resources and sticks to the characterized optimal basis.
Self Supervised Correlation-based Permutations for Multi-View Clustering
Venue: arXiv cs.LG 2025
Year: 2025

Combining data from different sources can improve data analysis tasks such as clustering. However, most of the current multi-view clustering methods are limited to specific domains or rely on a suboptimal and computationally intensive two-stage process of representation learning and clustering. We propose an end-to-end deep learning-based multi-view clustering framework for general data types (such as images and tables). Our approach involves generating meaningful fused representations using a novel permutation-based canonical correlation objective. We provide a theoretical analysis showing how the learned embeddings approximate those obtained by supervised linear discriminant analysis (LDA). Cluster assignments are learned by identifying consistent pseudo-labels across multiple views. Additionally, we establish a theoretical bound on the error caused by incorrect pseudo-labels in the unsupervised representations compared to LDA. Extensive experiments on ten multi-view clustering benchmark datasets provide empirical evidence for the effectiveness of the proposed model.
Tracking Semantic Change in Slovene: A Novel Dataset and Optimal Transport-Based Distance
Venue: arXiv cs.CL 2025
Year: 2025

In this paper, we focus on the detection of semantic changes in Slovene, a less resourced Slavic language with two million speakers. Detecting and tracking semantic changes provides insight into the evolution of language caused by changes in society and culture. We present the first Slovene dataset for evaluating semantic change detection systems, which contains aggregated semantic change scores for 104 target words obtained from more than 3,000 manually annotated sentence pairs. We analyze an important class of measures of semantic change metrics based on the Average pairwise distance and identify several limitations. To address these limitations, we propose a novel metric based on regularized optimal transport, which offers a more robust framework for quantifying semantic change. We provide a comprehensive evaluation of various existing semantic change detection methods and associated semantic change measures on our dataset. Through empirical testing, we demonstrate that our proposed approach, leveraging regularized optimal transport, achieves either matching or improved performance compared to baseline approaches.
GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoning
Venue: arXiv cs.AI 2025
Year: 2025

Generative Artificial Intelligence (GenAI) and communication networks are expected to have groundbreaking synergies for 6G. Connecting GenAI agents via a wireless network can potentially unleash the power of Collective Intelligence (CI) and pave the way for Artificial General Intelligence (AGI). However, current wireless networks are designed as a "data pipe" and are not suited to accommodate and leverage the power of GenAI. In this paper, we propose the GenAINet framework in which distributed GenAI agents communicate knowledge (facts, experiences, and methods) to accomplish arbitrary tasks. We first propose an architecture for a single GenAI agent and then provide a network architecture integrating GenAI capabilities to manage both network protocols and applications. Building on this, we investigate effective communication and reasoning problems by proposing a semantic-native GenAINet. Specifically, GenAI agents extract semantics from heterogeneous raw data, build and maintain a knowledge model representing the semantic relationships among pieces of knowledge, which is retrieved by GenAI models for planning and reasoning. Under this paradigm, different levels of collaboration can be achieved flexibly depending on the complexity of targeted tasks. Furthermore, we conduct two case studies in which, through wireless device queries, we demonstrate that extracting, compressing and transferring common knowledge can improve query accuracy while reducing communication costs; and in the wireless power control problem, we show that distributed agents can complete general tasks independently through collaborative reasoning without predefined communication protocols. Finally, we discuss challenges and future research directions in applying Large Language Models (LLMs) in 6G networks.
Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems
Venue: arXiv cs.AI 2025
Year: 2025

Understanding citizens' values in participatory systems is crucial for citizen-centric policy-making. We envision a hybrid participatory system where participants make choices and provide motivations for those choices, and AI agents estimate their value preferences by interacting with them. We focus on situations where a conflict is detected between participants' choices and motivations, and propose methods for estimating value preferences while addressing detected inconsistencies by interacting with the participants. We operationalize the philosophical stance that "valuing is deliberatively consequential." That is, if a participant's choice is based on a deliberation of value preferences, the value preferences can be observed in the motivation the participant provides for the choice. Thus, we propose and compare value preferences estimation methods that prioritize the values estimated from motivations over the values estimated from choices alone. Then, we introduce a disambiguation strategy that combines Natural Language Processing and Active Learning to address the detected inconsistencies between choices and motivations. We evaluate the proposed methods on a dataset of a large-scale survey on energy transition. The results show that explicitly addressing inconsistencies between choices and motivations improves the estimation of an individual's value preferences. The disambiguation strategy does not show substantial improvements when compared to similar baselines--however, we discuss how the novelty of the approach can open new research avenues and propose improvements to address the current limitations.