OpenRaiser/Intern-Atlas huggingface dataset:
Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles
Venue: arXiv cs.LG 2025
Year: 2025

Spurious correlations in the data, where multiple cues are predictive of the target labels, often lead to a phenomenon known as shortcut learning, where a model relies on erroneous, easy-to-learn cues while ignoring reliable ones. In this work, we propose DiffDiv an ensemble diversification framework exploiting Diffusion Probabilistic Models (DPMs) to mitigate this form of bias. We show that at particular training intervals, DPMs can generate images with novel feature combinations, even when trained on samples displaying correlated input features. We leverage this crucial property to generate synthetic counterfactuals to increase model diversity via ensemble disagreement. We show that DPM-guided diversification is sufficient to remove dependence on shortcut cues, without a need for additional supervised signals. We further empirically quantify its efficacy on several diversification objectives, and finally show improved generalization and diversification on par with prior work that relies on auxiliary data collection.
On the Robustness of Decision-Focused Learning
Venue: arXiv cs.LG 2025
Year: 2025

Decision-Focused Learning (DFL) is an emerging learning paradigm that tackles the task of training a machine learning (ML) model to predict missing parameters of an incomplete optimization problem, where the missing parameters are predicted. DFL trains an ML model in an end-to-end system, by integrating the prediction and optimization tasks, providing better alignment of the training and testing objectives. DFL has shown a lot of promise and holds the capacity to revolutionize decision-making in many real-world applications. However, very little is known about the performance of these models under adversarial attacks. We adopt ten unique DFL methods and benchmark their performance under two distinctly focused attacks adapted towards the Predict-then-Optimize problem setting. Our study proposes the hypothesis that the robustness of a model is highly correlated with its ability to find predictions that lead to optimal decisions without deviating from the ground-truth label. Furthermore, we provide insight into how to target the models that violate this condition and show how these models respond differently depending on the achieved optimality at the end of their training cycles.
A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Models
Venue: arXiv cs.CL 2025
Year: 2025

Dialogue systems (DS), including the task-oriented dialogue system (TOD) and the open-domain dialogue system (ODD), have always been a fundamental task in natural language processing (NLP), allowing various applications in practice. Owing to sophisticated training and well-designed model architecture, language models (LM) are usually adopted as the necessary backbone to build the dialogue system. Consequently, every breakthrough in LM brings about a shift in learning paradigm and research attention within dialogue system, especially the appearance of pre-trained language models (PLMs) and large language models (LLMs). In this paper, we take a deep look at the history of the dialogue system, especially its special relationship with the advancements of language models. Thus, our survey offers a systematic perspective, categorizing different stages in a chronological order aligned with LM breakthroughs, providing a comprehensive review of state-of-the-art research outcomes. What's more, we turn our attention to emerging topics and engage in a discussion on open challenges, providing valuable insights into the future directions for LLM-based dialogue systems. In summary, this survey delves into the dynamic interplay between language models and dialogue systems, unraveling the evolutionary path of this essential relationship. Through this exploration, we pave the way for a deeper comprehension of the field, guiding future developments in LM-based dialogue systems.
On the Convergence Rate of Linear Datalogo over Stable Semirings
Venue: arXiv cs.DB 2025
Year: 2025

Datalogo is an extension of Datalog, where instead of a program being a collection of union of conjunctive queries over the standard Boolean semiring, a program may now be a collection of sum-product queries over an arbitrary commutative partially ordered pre-semiring. Datalogo is more powerful than Datalog in that its additional algebraic structure alows for supporting recursion with aggregation. At the same time, Datalogo retains the syntactic and semantic simplicity of Datalog: Datalogo has declarative least fixpoint semantics. The least fixpoint can be found via the naïve evaluation algorithm that repeatedly applies the immediate consequence operator until no further change is possible. It was shown in~\cite{Khamis0PSW22} that, when the underlying semiring is $p$-stable, then the naïve evaluation of any Datalogo program over the semiring converges in a finite number of steps. However, the upper bounds on the rate of convergence were exponential in the number $n$ of ground IDB atoms. This paper establishes polynomial upper bounds on the convergence rate of the naïve algorithm on {\bf linear} Datalogo programs, which is quite common in practice. In particular, the main result of this paper is that the convergence rate of linear Datalogo programs under any $p$-stable semiring is $O(pn^3)$. Next, we study the convergence rate in terms of the number of elements in the semiring for linear Datalogo programs. When $L$ is the number of elements, we show that the convergence rate is bounded by $O(pn \log L)$. This significantly improves the convergence rate for small $L$.
Higher-Order DisCoCat (Peirce-Lambek-Montague semantics)
Venue: arXiv cs.CL 2025
Year: 2025

We propose a new definition of higher-order DisCoCat (categorical compositional distributional) models where the meaning of a word is not a diagram, but a diagram-valued higher-order function. Our models can be seen as a variant of Montague semantics based on a lambda calculus where the primitives act on string diagrams rather than logical formulae. As a special case, we show how to translate from the Lambek calculus into Peirce's system beta for first-order logic. This allows us to give a purely diagrammatic treatment of higher-order and non-linear processes in natural language semantics: adverbs, prepositions, negation and quantifiers. The definition presented in this article comes with a proof-of-concept implementation in DisCoPy, the Python library for string diagrams.
TOP-Former: A Multi-Agent Transformer Approach for the Team Orienteering Problem
Venue: arXiv cs.AI 2025
Year: 2025

Route planning for a fleet of vehicles is an important task in applications such as package delivery, surveillance, or transportation, often integrated within larger Intelligent Transportation Systems (ITS). This problem is commonly formulated as a Vehicle Routing Problem (VRP) known as the Team Orienteering Problem (TOP). Existing solvers for this problem primarily rely on either linear programming, which provides accurate solutions but requires computation times that grow with the size of the problem, or heuristic methods, which typically find suboptimal solutions in a shorter time. In this paper, we introduce TOP-Former, a multi-agent route planning neural network designed to efficiently and accurately solve the Team Orienteering Problem. The proposed algorithm is based on a centralized Transformer neural network capable of learning to encode the scenario (modeled as a graph) and analyze the complete context of all agents to deliver fast, precise, and collaborative solutions. Unlike other neural network-based approaches that adopt a more local perspective, TOP-Former is trained to understand the global situation of the vehicle fleet and generate solutions that maximize long-term expected returns. Extensive experiments demonstrate that the presented system outperforms most state-of-the-art methods in terms of both accuracy and computation speed.
Privacy Preserving Event Detection
Venue: arXiv cs.IT 2025
Year: 2025

This paper presents a privacy-preserving event detection scheme based on measurements made by a network of sensors. A diameter-like decision statistic made up of the marginal types of the measurements observed by the sensors is employed. The proposed detection scheme can achieve the best type-I error exponent as the type-II error rate is required to be negligible. Detection performance with finite-length observations is also demonstrated through a simulation example of spectrum sensing. Privacy protection is achieved by obfuscating the sensors' marginal types with random zero-modulo-sum numbers that are generated and distributed via the exchange of encrypted messages among the sensors. The privacy-preserving performance against "honest but curious" adversaries, including colluding sensors, the fusion center, and external eavesdroppers, is analyzed through a series of cryptographic games. It is shown that the probability that any probabilistic polynomial time adversary successfully estimates the sensors' measured types cannot be much better than independent guessing, when there are at least two non-colluding sensors.
A ripple in time: a discontinuity in American history
Venue: arXiv cs.CL 2025
Year: 2025

In this technical note we suggest a novel approach to discover temporal (related and unrelated to language dilation) and personality (authorship attribution) aspects in historical datasets. We exemplify our approach on the State of the Union addresses given by the past 42 US presidents: this dataset is known for its relatively small amount of data, and high variability of the size and style of texts. Nevertheless, we manage to achieve about 95\% accuracy on the authorship attribution task, and pin down the date of writing to a single presidential term.
Skeletal Cut Loci on Convex Polyhedra
Venue: arXiv cs.CG 2025
Year: 2025

On a convex polyhedron P, the cut locus C(x) with respect to a point x is a tree of geodesic segments (shortest paths) on P that includes every vertex. We say that P has a skeletal cut locus if there is some x in P such that C(x) subset Sk(P), where Sk(P) is the 1-skeleton of P. At a first glance, there seems to be very little relation between the cut locus and the 1-skeleton, as the first one is an intrinsic geometry notion, and the second one specifies the combinatorics of P. In this paper we study skeletal cut loci, obtaining four main results. First, given any combinatorial tree T, there exists a convex polyhedron P and a point x in P with a cut locus that lies in Sk(P), and whose combinatorics match T. Second, any (non-degenerate) polyhedron P has at most a finite number of points x for which C(x) subset Sk(P). Third, we show that almost all polyhedra have no skeletal cut locus. Fourth, we provide a combinatorial restriction to the existence of skeletal cut loci. Because the source unfolding of P with respect to x is always a non-overlapping net for P, and because the boundary of the source unfolding is the (unfolded) cut locus, source unfoldings of polyhedra with skeletal cut loci are edge-unfoldings, and moreover "blooming," avoiding self-intersection during an unfolding process. We also explore partially skeletal cut loci, leading to partial edge-unfoldings; i.e., unfoldings obtained by cutting along some polyhedron edges and cutting some non-edges.
Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
Venue: arXiv cs.CL 2025
Year: 2025

Both graph structures and textual information play a critical role in Knowledge Graph Completion (KGC). With the success of Pre-trained Language Models (PLMs) such as BERT, they have been applied for text encoding for KGC. However, the current methods mostly prefer to fine-tune PLMs, leading to huge training costs and limited scalability to larger PLMs. In contrast, we propose to utilize prompts and perform KGC on a frozen PLM with only the prompts trained. Accordingly, we propose a new KGC method named PDKGC with two prompts -- a hard task prompt which is to adapt the KGC task to the PLM pre-training task of token prediction, and a disentangled structure prompt which learns disentangled graph representation so as to enable the PLM to combine more relevant structure knowledge with the text information. With the two prompts, PDKGC builds a textual predictor and a structural predictor, respectively, and their combination leads to more comprehensive entity prediction. Solid evaluation on three widely used KGC datasets has shown that PDKGC often outperforms the baselines including the state-of-the-art, and its components are all effective. Our codes and data are available at https://github.com/genggengcss/PDKGC.
A Linear-time Simulation of Deterministic $d$-Limited Automata
Venue: arXiv cs.FL 2025
Year: 2025

A $d$-limited automaton is a Turing machine that may rewrite each input cell at most~$d$ times. Hibbard (1967) showed that for every $d \geq 2$ such automata recognize all context-free languages and that deterministic $d$-limited automata form a strict hierarchy. Later, Pighizzini and Pisoni proved that the second level of this hierarchy coincides with deterministic context-free languages (DCFLs). We present a linear-time recognition algorithm for deterministic $d$-limited automata in the RAM model, thereby extending linear-time recognition beyond DCFLs. We further generalize this result to deterministic $d(n)$-limited automata, where the bound $d$ may depend on the input length $n$. In addition, we prove an $O(n \cdot k \cdot d(n) + m)$ bound for the membership problem, where the input includes both the word and the automaton's description, with $m$ denoting the size of the description and $k$ the number of states.
Cascade Reinforcement Learning with State Space Factorization for O-RAN-based Traffic Steering
Venue: arXiv cs.NI 2025
Year: 2025

The Open Radio Access Network (O-RAN) architecture empowers intelligent and automated optimization of the RAN through applications deployed on the RAN Intelligent Controller (RIC) platform, enabling capabilities beyond what is achievable with traditional RAN solutions. Within this paradigm, Traffic Steering (TS) emerges as a pivotal RIC application that focuses on optimizing cell-level mobility settings in near-real-time, aiming to significantly improve network spectral efficiency. In this paper, we design a novel TS algorithm based on a Cascade Reinforcement Learning (CaRL) framework. We propose state space factorization and policy decomposition to reduce the need for large models and well-labeled datasets. For each sub-state space, an RL sub-policy will be trained to learn an optimized mapping onto the action space. To apply CaRL on new network regions, we propose a knowledge transfer approach to initialize a new sub-policy based on knowledge learned by the trained policies. To evaluate CaRL, we build a data-driven and scalable RIC digital twin (DT) that is modeled using important real-world data, including network configuration, user geo-distribution, and traffic demand, among others, from a tier-1 mobile operator in the US. We evaluate CaRL on two DT scenarios representing two network clusters in two different cities and compare its performance with the business-as-usual (BAU) policy and other competing optimization approaches using heuristic and Q-table algorithms. Benchmarking results show that CaRL performs the best and improves the average cluster-aggregated downlink throughput over the BAU policy by 24% and 18% in these two scenarios, respectively.
Shapley-Based Data Valuation with Mutual Information: A Key to Modified K-Nearest Neighbors
Venue: arXiv cs.LG 2025
Year: 2025

The K-Nearest Neighbors (KNN) algorithm is widely used for classification and regression; however, it suffers from limitations, including the equal treatment of all samples. We propose Information-Modified KNN (IM-KNN), a novel approach that leverages Mutual Information ($I$) and Shapley values to assign weighted values to neighbors, thereby bridging the gap in treating all samples with the same value and weight. On average, IM-KNN improves the accuracy, precision, and recall of traditional KNN by 16.80%, 17.08%, and 16.98%, respectively, across 12 benchmark datasets. Experiments on four large-scale datasets further highlight IM-KNN's robustness to noise, imbalanced data, and skewed distributions.
Hot PATE: Private Aggregation of Distributions for Diverse Task
Venue: arXiv cs.LG 2025
Year: 2025

The Private Aggregation of Teacher Ensembles (PATE) framework enables privacy-preserving machine learning by aggregating responses from disjoint subsets of sensitive data. Adaptations of PATE to tasks with inherent output diversity such as text generation, where the desired output is a sample from a distribution, face a core tension: as diversity increases, samples from different teachers are less likely to agree, but lower agreement results in reduced utility for the same privacy requirements. Yet suppressing diversity to artificially increase agreement is undesirable, as it distorts the output of the underlying model, and thus reduces output quality. We propose Hot PATE, a variant of PATE designed for diverse generative settings. We formalize the notion of a diversity-preserving ensemble sampler and introduce an efficient sampler that provably transfers diversity without incurring additional privacy cost. Hot PATE requires only API access to proprietary models and can be used as a drop-in replacement for existing Cold PATE samplers. Our empirical evaluations corroborate and quantify the benefits, showing significant improvements in the privacy utility trade-off on evaluated in-context learning tasks, both in preserving diversity and in returning relevant responses.
Meta-learning Optimizers for Communication-Efficient Learning
Venue: arXiv cs.LG 2025
Year: 2025

Communication-efficient variants of SGD, specifically local SGD, have received a great deal of interest in recent years. These approaches compute multiple gradient steps locally on each worker, before averaging model parameters, helping relieve the critical communication bottleneck in distributed deep learning training. Although many variants of these approaches have been proposed, they can sometimes lag behind state-of-the-art adaptive optimizers for deep learning. In this work, we investigate if the recent progress in the emerging area of learned optimizers can potentially close this gap in homogeneous data and homogeneous device settings while remaining communication-efficient. Specifically, we meta-learn how to perform global updates given an update from local SGD iterations. Our results demonstrate that learned optimizers can substantially outperform local SGD and its sophisticated variants while maintaining their communication efficiency. Our learned optimizers can even generalize to unseen and much larger datasets and architectures, including ImageNet and ViTs, and to unseen modalities such as language modeling. We therefore show the potential of learned optimizers for improving communication-efficient distributed learning.
Design and Evaluation of an Uncertainty-Aware Shared-Autonomy System with Hierarchical Conservative Skill Inference
Venue: arXiv cs.RO 2025
Year: 2025

Shared-autonomy imitation learning lets a human correct a robot in real time, mitigating covariate-shift errors. Yet existing approaches ignore two critical factors: (i) the operator's cognitive load and (ii) the risk created by delayed or erroneous interventions. We present an uncertainty-aware shared-autonomy system in which the robot modulates its behaviour according to a learned estimate of latent-space skill uncertainty. A hierarchical policy first infers a conservative skill embedding and then decodes it into low-level actions, enabling rapid task execution while automatically slowing down when uncertainty is high. We detail a full, open-source VR-teleoperation pipeline that is compatible with multi-configuration manipulators such as UR-series arms. Experiments on pouring and pick-and-place tasks demonstrate 70-90% success in dynamic scenes with moving targets, and a qualitative study shows a marked reduction in collision events compared with a non-conservative baseline. Although a dedicated ablation that isolates uncertainty is impractical on hardware for safety and cost reasons, the reported gains in stability and operator workload already validate the design and motivate future large-scale studies.
Towards Automated Discovery of Asymmetric Mempool DoS in Blockchains
Venue: arXiv cs.CR 2025
Year: 2025

In blockchains, mempool controls transaction flow before consensus, denial of whose service hurts the health and security of blockchain networks. This paper presents MPFUZZ, the first mempool fuzzer to find asymmetric DoS bugs by exploring the space of symbolized mempool states and optimistically estimating the promisingness of an intermediate state in reaching bug oracles. Compared to the baseline blockchain fuzzers, MPFUZZ achieves a > 100x speedup in finding known DETER exploits. Running MPFUZZ on major Ethereum clients leads to discovering new mempool vulnerabilities, which exhibit a wide variety of sophisticated patterns, including stealthy mempool eviction and mempool locking. Rule-based mitigation schemes are proposed against all newly discovered vulnerabilities.
Airdrops: Giving Money Away Is Harder Than It Seems
Venue: arXiv cs.CR 2025
Year: 2025

Airdrops are a popular mechanism used by blockchain protocols to bootstrap communities, reward early adopters, and decentralize token distribution. Despite their widespread adoption, the effectiveness of airdrops in achieving long-term user engagement and ecosystem growth remains poorly understood. In this paper, we present the first comprehensive empirical study of nine major airdrops across Ethereum and Layer-2 ecosystems. Our analysis reveals that a substantial share of tokens--up to 66% in some cases--are rapidly sold, often in recipients' first post-claim transaction. We show that this behavior is largely driven by "airdrop farmers," who strategically optimize eligibility criteria to extract value without contributing meaningfully to the ecosystem. We complement our quantitative findings with a case study of the Arbitrum airdrop, illustrating how short-term activity spikes fail to translate into sustained user involvement. Based on these results, we discuss common design pitfalls--such as Sybil vulnerability, poor incentive alignment, and governance token misuse--and propose actionable guidelines for designing more effective airdrop strategies.
Energy-consistent integration of mechanical systems based on Livens principle
Venue: arXiv cs.CE 2025
Year: 2025

In this work we make use of Livens principle (sometimes also referred to as Hamilton-Pontryagin principle) in order to obtain a novel structure-preserving integrator for mechanical systems. In contrast to the canonical Hamiltonian equations of motion, the Euler-Lagrange equations pertaining to Livens principle circumvent the need to invert the mass matrix. This is an essential advantage with respect to singular mass matrices, which can yield severe difficulties for the modelling and simulation of multibody systems. Moreover, Livens principle unifies both Lagrangian and Hamiltonian viewpoints on mechanics. Additionally, the present framework avoids the need to set up the system's Hamiltonian. The novel scheme algorithmically conserves a general energy function and aims at the preservation of momentum maps corresponding to symmetries of the system. We present an extension to mechanical systems subject to holonomic constraints. The performance of the newly devised method is studied in representative examples.