OpenRaiser/Intern-Atlas huggingface dataset:
Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning
Venue: arXiv cs.LG 2025
Year: 2025

We study centralized distributed data parallel training of deep neural networks (DNNs), aiming to improve the trade-off between communication efficiency and model performance of the local gradient methods. To this end, we revisit the flat-minima hypothesis, which suggests that models with better generalization tend to lie in flatter regions of the loss landscape. We introduce a simple, yet effective, sharpness measure, Inverse Mean Valley, and demonstrate its strong correlation with the generalization gap of DNNs. We incorporate an efficient relaxation of this measure into the distributed training objective as a lightweight regularizer that encourages workers to collaboratively seek wide minima. The regularizer exerts a pushing force that counteracts the consensus step pulling the workers together, giving rise to the Distributed Pull-Push Force (DPPF) algorithm. Empirically, we show that DPPF outperforms other communication-efficient approaches and achieves better generalization performance than local gradient methods and synchronous gradient averaging, while maintaining communication efficiency. In addition, our loss landscape visualizations confirm the ability of DPPF to locate flatter minima. On the theoretical side, we show that DPPF guides workers to span flat valleys, with the final valley width governed by the interplay between push and pull strengths, and that its pull-push dynamics is self-stabilizing. We further provide generalization guarantees linked to the valley width and prove convergence in the non-convex setting.
Teleoperating Autonomous Vehicles over Commercial 5G Networks: Are We There Yet?
Venue: arXiv cs.NI 2025
Year: 2025

Remote driving, or teleoperating Autonomous Vehicles (AVs), is a key application that emerging 5G networks aim to support. In this paper, we conduct a systematic feasibility study of AV teleoperation over commercial 5G networks from both cross-layer and end-to-end (E2E) perspectives. Given the critical importance of timely delivery of sensor data, such as camera and LiDAR data, for AV teleoperation, we focus in particular on the performance of uplink sensor data delivery. We analyze the impacts of Physical Layer (PHY layer) 5G radio network factors, including channel conditions, radio resource allocation, and Handovers (HOs), on E2E latency performance. We also examine the impacts of 5G networks on the performance of upper-layer protocols and E2E application Quality-of-Experience (QoE) adaptation mechanisms used for real-time sensor data delivery, such as Real-Time Streaming Protocol (RTSP) and Web Real Time Communication (WebRTC). Our study reveals the challenges posed by today's 5G networks and the limitations of existing sensor data streaming mechanisms. The insights gained will help inform the co-design of future-generation wireless networks, edge cloud systems, and applications to overcome the low-latency barriers in AV teleoperation.
STARN-GAT: A Multi-Modal Spatio-Temporal Graph Attention Network for Accident Severity Prediction
Venue: arXiv cs.AI 2025
Year: 2025

Accurate prediction of traffic accident severity is critical for improving road safety, optimizing emergency response strategies, and informing the design of safer transportation infrastructure. However, existing approaches often struggle to effectively model the intricate interdependencies among spatial, temporal, and contextual variables that govern accident outcomes. In this study, we introduce STARN-GAT, a Multi-Modal Spatio-Temporal Graph Attention Network, which leverages adaptive graph construction and modality-aware attention mechanisms to capture these complex relationships. Unlike conventional methods, STARN-GAT integrates road network topology, temporal traffic patterns, and environmental context within a unified attention-based framework. The model is evaluated on the Fatality Analysis Reporting System (FARS) dataset, achieving a Macro F1-score of 85 percent, ROC-AUC of 0.91, and recall of 81 percent for severe incidents. To ensure generalizability within the South Asian context, STARN-GAT is further validated on the ARI-BUET traffic accident dataset, where it attains a Macro F1-score of 0.84, recall of 0.78, and ROC-AUC of 0.89. These results demonstrate the model's effectiveness in identifying high-risk cases and its potential for deployment in real-time, safety-critical traffic management systems. Furthermore, the attention-based architecture enhances interpretability, offering insights into contributing factors and supporting trust in AI-assisted decision-making. Overall, STARN-GAT bridges the gap between advanced graph neural network techniques and practical applications in road safety analytics.
Your Attention Matters: to Improve Model Robustness to Noise and Spurious Correlations
Venue: arXiv cs.LG 2025
Year: 2025

Self-attention mechanisms are foundational to Transformer architectures, supporting their impressive success in a wide range of tasks. While there are many self-attention variants, their robustness to noise and spurious correlations has not been well studied. This study evaluates Softmax, Sigmoid, Linear, Doubly Stochastic, and Cosine attention within Vision Transformers under different data corruption scenarios. Through testing across the CIFAR-10, CIFAR-100, and Imagenette datasets, we show that Doubly Stochastic attention is the most robust. It consistently outperformed the next best mechanism by $0.1\%-5.1\%$ when training data, or both training and testing data, were corrupted. Our findings inform self-attention selection in contexts with imperfect data. The code used is available at https://github.com/ctamayor/NeurIPS-Robustness-ViT.
Conditional Diffusion Models for Global Precipitation Map Inpainting
Venue: arXiv cs.LG 2025
Year: 2025

Incomplete satellite-based precipitation presents a significant challenge in global monitoring. For example, the Global Satellite Mapping of Precipitation (GSMaP) from JAXA suffers from substantial missing regions due to the orbital characteristics of satellites that have microwave sensors, and its current interpolation methods often result in spatial discontinuities. In this study, we formulate the completion of the precipitation map as a video inpainting task and propose a machine learning approach based on conditional diffusion models. Our method employs a 3D U-Net with a 3D condition encoder to reconstruct complete precipitation maps by leveraging spatio-temporal information from infrared images, latitude-longitude grids, and physical time inputs. Training was carried out on ERA5 hourly precipitation data from 2020 to 2023. We generated a pseudo-GSMaP dataset by randomly applying GSMaP masks to ERA maps. Performance was evaluated for the calendar year 2024, and our approach produces more spatio-temporally consistent inpainted precipitation maps compared to conventional methods. These results indicate the potential to improve global precipitation monitoring using the conditional diffusion models.
Large-Scale LiDAR-Inertial Dataset for Degradation-Robust High-Precision Mapping
Venue: arXiv cs.RO 2025
Year: 2025

This paper introduces a large-scale, high-precision LiDAR-Inertial Odometry (LIO) dataset, aiming to address the insufficient validation of LIO systems in complex real-world scenarios in existing research. The dataset covers four diverse real-world environments spanning 60,000 to 750,000 square meters, collected using a custom backpack-mounted platform equipped with multi-beam LiDAR, an industrial-grade IMU, and RTK-GNSS modules. The dataset includes long trajectories, complex scenes, and high-precision ground truth, generated by fusing SLAM-based optimization with RTK-GNSS anchoring, and validated for trajectory accuracy through the integration of oblique photogrammetry and RTK-GNSS. This dataset provides a comprehensive benchmark for evaluating the generalization ability of LIO systems in practical high-precision mapping scenarios.
Enhancing Hallucination Detection via Future Context
Venue: arXiv cs.CL 2025
Year: 2025

Large Language Models (LLMs) are widely used to generate plausible text on online platforms, without revealing the generation process. As users increasingly encounter such black-box outputs, detecting hallucinations has become a critical challenge. To address this challenge, we focus on developing a hallucination detection framework for black-box generators. Motivated by the observation that hallucinations, once introduced, tend to persist, we sample future contexts. The sampled future contexts provide valuable clues for hallucination detection and can be effectively integrated with various sampling-based methods. We extensively demonstrate performance improvements across multiple methods using our proposed sampling approach.
Construction of non-generalized Reed-Solomon MDS codes based on systematic generator matrix
Venue: arXiv cs.IT 2025
Year: 2025

Maximum distance separable (MDS) codes are considered optimal because the minimum distance cannot be improved for a given length and code size. The most prominent MDS codes are likely the generalized Reed-Solomon (GRS) codes. In 1989, Roth and Lempel constructed a type of MDS code that is not a GRS code (referred to as non-GRS). In 2017, Beelen et al. introduced twisted Reed-Solomon (TRS) codes and demonstrated that many MDS TRS codes are indeed non-GRS. Following this, the definition of TRS codes was generalized to the most comprehensive form, which we refer to as generalized twisted Reed-Solomon (GTRS) codes. In this paper, we prove that two families of GTRS codes are non-GRS and provide a systematic generator matrix for a class of GTRS codes. Inspired by the form of the systematic generator matrix for GTRS codes,we also present a construction of non-GRS MDS codes.
DAG-AFL:Directed Acyclic Graph-based Asynchronous Federated Learning
Venue: arXiv cs.LG 2025
Year: 2025

Due to the distributed nature of federated learning (FL), the vulnerability of the global model and the need for coordination among many client devices pose significant challenges. As a promising decentralized, scalable and secure solution, blockchain-based FL methods have attracted widespread attention in recent years. However, traditional consensus mechanisms designed for Proof of Work (PoW) similar to blockchain incur substantial resource consumption and compromise the efficiency of FL, particularly when participating devices are wireless and resource-limited. To address asynchronous client participation and data heterogeneity in FL, while limiting the additional resource overhead introduced by blockchain, we propose the Directed Acyclic Graph-based Asynchronous Federated Learning (DAG-AFL) framework. We develop a tip selection algorithm that considers temporal freshness, node reachability and model accuracy, with a DAG-based trusted verification strategy. Extensive experiments on 3 benchmarking datasets against eight state-of-the-art approaches demonstrate that DAG-AFL significantly improves training efficiency and model accuracy by 22.7% and 6.5% on average, respectively.
A note on the Artstein-Avidan-Milman's generalized Legendre transforms
Venue: arXiv cs.IT 2025
Year: 2025

Artstein-Avidan and Milman [Annals of mathematics (2009), (169):661-674] characterized invertible reverse-ordering transforms on the space of lower semi-continuous extended real-valued convex functions as affine deformations of the ordinary Legendre transform. In this work, we first prove that all those generalized Legendre transforms on functions correspond to the ordinary Legendre transform on dually corresponding affine-deformed functions: In short, generalized convex conjugates are ordinary convex conjugates of dually affine-deformed functions. Second, we explain how these generalized Legendre transforms can be derived from the dual Hessian structures of information geometry.
Methods for the Segmentation of Reticular Structures Using 3D LiDAR Data: A Comparative Evaluation
Venue: arXiv cs.RO 2025
Year: 2025

Reticular structures form the backbone of major infrastructure like bridges, pylons, and airports, but their inspection and maintenance are costly and hazardous, often requiring human intervention. While prior research has focused on fault detection via images or robotic platform design, the autonomous navigation of robots within these structures is less explored. This study addresses that gap by proposing methods to detect navigable surfaces in truss structures, enhancing the autonomy of climbing robots. The paper introduces several approaches for binary segmentation of navigable surfaces versus background from 3D point clouds of metallic trusses. These methods fall into two categories: analytical algorithms and deep learning models. The analytical approach features a custom algorithm that segments structures by analyzing the eigendecomposition of planar patches in the point cloud. In parallel, advanced deep learning models PointNet, PointNet++, MinkUNet34C, and PointTransformerV3 are trained and evaluated for the same task. Comparative analysis shows that the analytical algorithm offers easier parameter tuning and performance comparable to deep learning models, which, while more computationally intensive, excel in segmentation accuracy. Notably, PointTransformerV3 achieves a Mean Intersection Over Union (mIoU) of about 97%. The study demonstrates the promise of both analytical and deep learning methods for improving autonomous navigation in complex truss environments. The results highlight the trade-offs between computational efficiency and segmentation performance, providing valuable guidance for future research and practical applications in autonomous infrastructure inspection and maintenance.
The Random Variables of the DNA Coverage Depth Problem
Venue: arXiv cs.IT 2025
Year: 2025

DNA data storage systems encode digital data into DNA strands, enabling dense and durable storage. Efficient data retrieval depends on coverage depth, a key performance metric. We study the random access coverage depth problem and focus on minimizing the expected number of reads needed to recover information strands encoded via a linear code. We compute the asymptotic performance of a recently proposed code construction, establishing and refining a conjecture in the field by giving two independent proofs. We also analyze a geometric code construction based on balanced quasi-arcs and optimize its parameters. Finally, we investigate the full distribution of the random variables that arise in the coverage depth problem, of which the traditionally studied expectation is just the first moment. This allows us to distinguish between code constructions that, at first glance, may appear to behave identically.
Hot-Swap MarkBoard: An Efficient Black-box Watermarking Approach for Large-scale Model Distribution
Venue: arXiv cs.CR 2025
Year: 2025

Recently, Deep Learning (DL) models have been increasingly deployed on end-user devices as On-Device AI, offering improved efficiency and privacy. However, this deployment trend poses more serious Intellectual Property (IP) risks, as models are distributed on numerous local devices, making them vulnerable to theft and redistribution. Most existing ownership protection solutions (e.g., backdoor-based watermarking) are designed for cloud-based AI-as-a-Service (AIaaS) and are not directly applicable to large-scale distribution scenarios, where each user-specific model instance must carry a unique watermark. These methods typically embed a fixed watermark, and modifying the embedded watermark requires retraining the model. To address these challenges, we propose Hot-Swap MarkBoard, an efficient watermarking method. It encodes user-specific $n$-bit binary signatures by independently embedding multiple watermarks into a multi-branch Low-Rank Adaptation (LoRA) module, enabling efficient watermark customization without retraining through branch swapping. A parameter obfuscation mechanism further entangles the watermark weights with those of the base model, preventing removal without degrading model performance. The method supports black-box verification and is compatible with various model architectures and DL tasks, including classification, image generation, and text generation. Extensive experiments across three types of tasks and six backbone models demonstrate our method's superior efficiency and adaptability compared to existing approaches, achieving 100\% verification accuracy.
A Multimodal Architecture for Endpoint Position Prediction in Team-based Multiplayer Games
Venue: arXiv cs.CV 2025
Year: 2025

Understanding and predicting player movement in multiplayer games is crucial for achieving use cases such as player-mimicking bot navigation, preemptive bot control, strategy recommendation, and real-time player behavior analytics. However, the complex environments allow for a high degree of navigational freedom, and the interactions and team-play between players require models that make effective use of the available heterogeneous input data. This paper presents a multimodal architecture for predicting future player locations on a dynamic time horizon, using a U-Net-based approach for calculating endpoint location probability heatmaps, conditioned using a multimodal feature encoder. The application of a multi-head attention mechanism for different groups of features allows for communication between agents. In doing so, the architecture makes efficient use of the multimodal game state including image inputs, numerical and categorical features, as well as dynamic game data. Consequently, the presented technique lays the foundation for various downstream tasks that rely on future player positions such as the creation of player-predictive bot behavior or player anomaly detection.
Automated Catamorphism Synthesis for Solving Constrained Horn Clauses over Algebraic Data Types
Venue: arXiv cs.LO 2025
Year: 2025

We propose a novel approach to satisfiability checking of Constrained Horn Clauses (CHCs) over Algebraic Data Types (ADTs). CHC-based automated verification has gained considerable attention in recent years, leading to the development of various CHC solvers. However, existing solvers for CHCs over ADTs are not fully satisfactory, due to their limited ability to find and express models involving inductively defined functions/predicates (e.g., those about the sum of list elements). To address this limitation, we consider catamorphisms (generalized fold functions), and present a framework for automatically discovering appropriate catamorphisms on demand and using them to express a model of given CHCs. We have implemented a new CHC solver called Catalia based on the proposed method. Our experimental results for the CHC-COMP 2024 benchmark show that Catalia outperforms state-of-the-art solvers in solving satisfiable CHCs over ADTs. Catalia was also used as a core part of the tool called ChocoCatalia, which won the ADT-LIA category of CHC-COMP 2025.
Dark Side of Modalities: Reinforced Multimodal Distillation for Multimodal Knowledge Graph Reasoning
Venue: arXiv cs.MM 2025
Year: 2025

The multimodal knowledge graph reasoning (MKGR) task aims to predict the missing facts in the incomplete MKGs by leveraging auxiliary images and descriptions of entities. Existing approaches are trained with single-target objectives, which neglect the probabilistic correlations of entity labels, especially in non-target entities. Moreover, previous studies incorporate all modalities statically or adaptively, overlooking the negative impacts of irrelevant or misleading information in the incompetent modalities. To address these issues, we introduce a novel Reinforced Multimodal Distillation framework, exploiting the Dark Side of Modalities (DSoM) from two perspectives: (1) Dark knowledge from non-target entities: We propose to train a unimodal KGR model through logit distillation to mimic the multimodal soft labels provided by pre-trained multimodal teacher models. The multimodal soft labels could provide rich supervision signals with subtle correlations among both target and non-target entities from multiple perspectives. We further decouple logits into neighbor entities and non-neighbor entities to divide into two types of correlations. (2) Dark side in unhelpful modalities: To exclude the adverse effects of unhelpful modalities, we introduce a reinforced teacher combination mechanism that dynamically selects the optimal set of multimodal teachers for each triple. The agent is trained to maximize the rewards, which are only assigned to the beneficial multimodal combination strategies for the student model. Comprehensive experiments demonstrate the effectiveness of DSoM framework on 5 MKGR datasets. Codes are available at github.com/OreOZhao/DSoM.
Regularizing Subspace Redundancy of Low-Rank Adaptation
Venue: arXiv cs.CV 2025
Year: 2025

Low-Rank Adaptation (LoRA) and its variants have delivered strong capability in Parameter-Efficient Transfer Learning (PETL) by minimizing trainable parameters and benefiting from reparameterization. However, their projection matrices remain unrestricted during training, causing high representation redundancy and diminishing the effectiveness of feature adaptation in the resulting subspaces. While existing methods mitigate this by manually adjusting the rank or implicitly applying channel-wise masks, they lack flexibility and generalize poorly across various datasets and architectures. Hence, we propose ReSoRA, a method that explicitly models redundancy between mapping subspaces and adaptively Regularizes Subspace redundancy of Low-Rank Adaptation. Specifically, it theoretically decomposes the low-rank submatrices into multiple equivalent subspaces and systematically applies de-redundancy constraints to the feature distributions across different projections. Extensive experiments validate that our proposed method consistently facilitates existing state-of-the-art PETL methods across various backbones and datasets in vision-language retrieval and standard visual classification benchmarks. Besides, as a training supervision, ReSoRA can be seamlessly integrated into existing approaches in a plug-and-play manner, with no additional inference costs. Code is publicly available at: https://github.com/Lucenova/ReSoRA.
Industry Insights from Comparing Deep Learning and GBDT Models for E-Commerce Learning-to-Rank
Venue: arXiv cs.IR 2025
Year: 2025

In e-commerce recommender and search systems, tree-based models, such as LambdaMART, have set a strong baseline for Learning-to-Rank (LTR) tasks. Despite their effectiveness and widespread adoption in industry, the debate continues whether deep neural networks (DNNs) can outperform traditional tree-based models in this domain. To contribute to this discussion, we systematically benchmark DNNs against our production-grade LambdaMART model. We evaluate multiple DNN architectures and loss functions on a proprietary dataset from OTTO and validate our findings through an 8-week online A/B test. The results show that a simple DNN architecture outperforms a strong tree-based baseline in terms of total clicks and revenue, while achieving parity in total units sold.