OpenRaiser/Intern-Atlas huggingface dataset:
Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models
Venue: arXiv cs.LG 2025
Year: 2025

In the foreseeable future, autonomous vehicles will require human assistance in situations they can not resolve on their own. In such scenarios, remote assistance from a human can provide the required input for the vehicle to continue its operation. Typical sensors used in autonomous vehicles include camera and lidar sensors. Due to the massive volume of sensor data that must be sent in real-time, highly efficient data compression is elementary to prevent an overload of network infrastructure. Sensor data compression using deep generative neural networks has been shown to outperform traditional compression approaches for both image and lidar data, regarding compression rate as well as reconstruction quality. However, there is a lack of research about the performance of generative-neural-network-based compression algorithms for remote assistance. In order to gain insights into the feasibility of deep generative models for usage in remote assistance, we evaluate state-of-the-art algorithms regarding their applicability and identify potential weaknesses. Further, we implement an online pipeline for processing sensor data and demonstrate its performance for remote assistance using the CARLA simulator.
Modular Decomposition of Hierarchical Finite State Machines
Venue: arXiv cs.FL 2025
Year: 2025

Hierarchical Finite State Machines (HFSMs) are a standard software-modelling concept which extends the classical Finite State Machine (FSM) notion with the useful abstraction of hierarchical nesting. That is, an HFSM is an FSM whose states can be other FSMs. The hierarchy in HFSMs is provided at design time, and can be removed by expanding nested states, allowing HFSMs to inherit the semantics of FSMs. However, because hierarchy is a useful representation of the structure of an FSM, we would like to be able to invert this operation: given an FSM, can we compute equivalent HFSMs? This is the topic of this paper. By adapting the analogous theory of `modular decomposition' from graph theory into automata theory, we are able to compute an efficient representation of the space of equivalent HFSMs to a given one. Specifically, we first define a module of an FSM, which is a collection of nodes which can be treated as a nested FSM. Unlike modules in graphs, some modules in FSMs are lacking in algebraic structure. We identify a simple and natural restriction of the modules, called thin modules, which regain many of the critical properties from modules in graphs. We then construct a linear-space directed graph which uniquely represents every thin module, and hence every equivalent (thin) HFSM. We call this graph the modular decomposition. The modular decomposition makes clear the significant common structure underlying equivalent thin HFSMs. We provide an $O(n^2k)$ algorithm for constructing the modular decomposition of an $n$-state $k$-symbol FSM. We demonstrate the applicability of this theory on the following `bottleneck' problem: given an HFSM, find an equivalent one where the size of the largest component FSM is minimised. The modular decomposition gives a simple greedy algorithm for the bottleneck problem on thin HFSMs, which we demonstrate on a wristwatch HFSM example from Harel (1987).
On The Complexity of Maximizing Temporal Reachability via Trip Temporalisation
Venue: arXiv cs.DM 2025
Year: 2025

We consider the problem of assigning appearing times to the edges of a digraph in order to maximize the (average) temporal reachability between pairs of nodes. Motivated by the application to public transit networks, where edges cannot be scheduled independently one of another, we consider the setting where the edges are grouped into certain walks (called trips) in the digraph and where assigning the appearing time to the first edge of a trip forces the appearing times of the subsequent edges. In this setting, we show that, quite surprisingly, it is NP-complete to decide whether there exists an assignment of times connecting a given pair of nodes. This result allows us to prove that the problem of maximising the temporal reachability cannot be approximated within a factor better than some polynomial term in the size of the graph. We thus focus on the case where, for each pair of nodes, there exists an assignment of times such that one node is reachable from the other. We call this property strong temporalisability. It is a very natural assumption for the application to public transit networks. On the negative side, the problem of maximising the temporal reachability remains hard to approximate within a factor $\sqrt$ n/12 in that setting. Moreover, we show the existence of collections of trips that are strongly temporalisable but for which any assignment of starting times to the trips connects at most an O(1/ $\sqrt$ n) fraction of all pairs of nodes. On the positive side, we show that there must exist an assignment of times that connects a constant fraction of all pairs in the strongly temporalisable and symmetric case, that is, when the set of trips to be scheduled is such that, for each trip, there is a symmetric trip visiting the same nodes in reverse order. Keywords:edge labeling edge scheduled network network optimisation temporal graph temporal path temporal reachability time assignment
Distributed Evaluation of Graph Queries using Recursive Relational Algebra
Venue: arXiv cs.DB 2025
Year: 2025

We present a system called Dist-$μ$-RA for the distributed evaluation of recursive graph queries. Dist-$μ$-RA builds on the recursive relational algebra and extends it with evaluation plans suited for the distributed setting. The goal is to offer expressivity for high-level queries while providing efficiency at scale and reducing communication costs. Experimental results on both real and synthetic graphs show the effectiveness of the proposed approach compared to existing systems.
Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systems
Venue: arXiv cs.IR 2025
Year: 2025

A key distinguishing feature of conversational recommender systems over traditional recommender systems is their ability to elicit user preferences using natural language. Currently, the predominant approach to preference elicitation is to ask questions directly about items or item attributes. Users searching for recommendations may not have deep knowledge of the available options in a given domain. As such, they might not be aware of key attributes or desirable values for them. However, in many settings, talking about the planned use of items does not present any difficulties, even for those that are new to a domain. In this paper, we propose a novel approach to preference elicitation by asking implicit questions based on item usage. As one of the main contributions of this work, we develop a multi-stage data annotation protocol using crowdsourcing, to create a high-quality labeled training dataset. Another main contribution is the development of four models for the question generation task: two template-based baseline models and two neural text-to-text models. The template-based models use heuristically extracted common patterns found in the training data, while the neural models use the training data to learn to generate questions automatically. Using common metrics from machine translation for automatic evaluation, we show that our approaches are effective in generating elicitation questions, even with limited training data. We further employ human evaluation for comparing the generated questions using both pointwise and pairwise evaluation designs. We find that the human evaluation results are consistent with the automatic ones, allowing us to draw conclusions about the quality of the generated questions with certainty. Finally, we provide a detailed analysis of cases where the models show their limitations.
A more efficient algorithm to compute the Rand Index for change-point problems
Venue: arXiv cs.DS 2025
Year: 2025

We provide a more efficient algorithm for computing the Rand Index when the data cluster comes from a change-point detection problem. Given $N$ data points and two clusterings of size $r$ and $s$, the algorithm runs on $O(r+s)$ time complexity and $O(1)$ memory complexity. The traditional algorithm, in contrast, runs on $O(rs+N)$ time complexity and $O(rs)$ memory complexity.
On the Parallel Complexity of Group Isomorphism via Weisfeiler-Leman
Venue: arXiv cs.DS 2025
Year: 2025

In this paper, we show that the constant-dimensional Weisfeiler-Leman algorithm for groups (Brachter & Schweitzer, LICS 2020) can be fruitfully used to improve parallel complexity upper bounds on isomorphism testing for several families of groups. In particular, we show: - Groups with an Abelian normal Hall subgroup whose complement is $O(1)$-generated are identified by constant-dimensional Weisfeiler-Leman using only a constant number of rounds. This places isomorphism testing for this family of groups into $\textsf{L}$; the previous upper bound for isomorphism testing was $\textsf{P}$ (Qiao, Sarma, & Tang, STACS 2011). - We use the individualize-and-refine paradigm to obtain an isomorphism test for groups without Abelian normal subgroups by $\textsf{SAC}$ circuits of depth $O(\log n)$ and size $n^{O(\log \log n)}$, previously only known to be in $\textsf{P}$ (Babai, Codenotti, \& Qiao, ICALP 2012) and $\mathsf{quasiSAC}^1$ (Chattopadhyay, Torán, \& Wagner, ACM Trans. Comput. Theory, 2013). - We extend a result of Brachter \& Schweitzer (ESA, 2022) on direct products of groups to the parallel setting. Namely, we also show that Weisfeiler--Leman can identify direct products in parallel, provided it can identify each of the indecomposable direct factors in parallel. They previously showed the analogous result for $\textsf{P}$. We finally consider the count-free Weisfeiler--Leman algorithm, where we show that count-free WL is unable to even distinguish Abelian groups in polynomial-time. Nonetheless, we use count-free WL in tandem with bounded non-determinism and limited counting to obtain a new upper bound of $β_{1}\textsf{MAC}^{0}(\textsf{FOLL})$ for isomorphism testing of Abelian groups. This improves upon the previous $\textsf{TC}^{0}(\textsf{FOLL})$ upper bound due to Chattopadhyay, Torán, \& Wagner (ibid.).
Minimizing $\ell_2$ Norm of Flow Time by Starvation Mitigation
Venue: arXiv cs.DS 2025
Year: 2025

The assessment of a job's Quality of Service (QoS) often revolves around its flow time, also referred to as response time. This study delves into two fundamental objectives for scheduling jobs: the average flow time and the maximum flow time. While the Shortest Remaining Processing Time (SRPT) algorithm minimizes average flow time, it can result in job starvation, causing certain jobs to experience disproportionately long and unfair flow times. In contrast, the First-Come-First-Served (FCFS) algorithm minimizes the maximum flow time but may compromise the average flow time. To strike a balance between these two objectives, a common approach is to minimize the $\ell_2$ norm of flow time. SRPT and FCFS are $O(n^{\frac{1}{2}})$-competitive for this problem, where $n$ is the number of jobs. Prior to this work, no algorithm is known to achieve a competitive ratio better than SRPT and FCFS. In this paper, we use FCFS to mitigate the starvation caused by SRPT. Given a good estimate of $n$, we prove that this approach achieves a much better competitive ratio of $O(n^{\frac{1}{3}})$. Our results provide the first theoretical evidence that mitigating starvation in SRPT leads to a provable improvement in scheduling performance.
Space Complexity of Vertex Connectivity Oracles
Venue: arXiv cs.DS 2025
Year: 2025

A $k$-vertex connectivity oracle for undirected $G$ is a data structure that, given $u,v\in V(G)$, reports $\min\{k,κ(u,v)\}$, where $κ(u,v)$ is the pairwise vertex connectivity between $u,v$. There are three main measures of efficiency: construction time, query time, and space. Prior work of Izsak and Nutov shows that a data structure of total size $\tilde{O}(kn)$ can even be encoded as a $\tilde{O}(k)$-bit labeling scheme so that vertex-connectivity queries can be answered in $\tilde{O}(k)$ time. The construction time is polynomial, but unspecified. In this paper we address the top three complexity measures: Space, Query Time, and Construction Time. We give an $Ω(kn)$-bit lower bound on any vertex connectivity oracle. We construct an optimal-space connectivity oracle in max-flow time that answers queries in $O(\log n)$ time, independent of $k$.
Fast Decoding of Interleaved Linearized Reed-Solomon Codes and Variants
Venue: arXiv cs.IT 2025
Year: 2025

We construct $s$-interleaved linearized Reed--Solomon (ILRS) codes and variants and propose efficient decoding schemes that can correct errors beyond the unique decoding radius in the sum-rank metric. The proposed interpolation-based scheme for ILRS codes can be used as a list decoder or as a probabilistic unique decoder that corrects errors of sum-rank up to $t\leq\frac{s}{s+1}(n-k)$, where $s$ is the interleaving order, $n$ the length and $k$ the dimension of the code. Upper bounds on the list size and the decoding failure probability are given where the latter is based on a novel Loidreau--Overbeck-like decoder for ILRS codes. We show how the proposed decoding schemes can be used to decode errors beyond the unique decoding radius in the skew metric by using an isometry between the sum-rank metric and the skew metric. We generalize fast minimal approximant basis interpolation techniques to obtain efficient decoding schemes for ILRS codes (and variants) with subquadratic complexity in the code length. Up to our knowledge, the presented decoding schemes are the first being able to correct errors beyond the unique decoding region in the sum-rank and skew metric. The performance of the proposed decoding schemes and the tightness of the upper bound on the decoding failure probability are validated via Monte Carlo simulations.
Distortion-Aware Brushing for Reliable Cluster Analysis in Multidimensional Projections
Venue: arXiv cs.HC 2025
Year: 2025

Brushing is a common interaction technique in 2D scatterplots, allowing users to select clustered points within a continuous, enclosed region for further analysis or filtering. However, applying conventional brushing to 2D representations of multidimensional (MD) data, i.e., Multidimensional Projections (MDPs), can lead to unreliable cluster analysis due to MDP-induced distortions that inaccurately represent the cluster structure of the original MD data. To alleviate this problem, we introduce a novel brushing technique for MDPs called Distortion-aware brushing. As users perform brushing, Distortion-aware brushing corrects distortions around the currently brushed points by dynamically relocating points in the projection, pulling data points close to the brushed points in MD space while pushing distant ones apart. This dynamic adjustment helps users brush MD clusters more accurately, leading to more reliable cluster analysis. Our user studies with 24 participants show that Distortion-aware brushing significantly outperforms previous brushing techniques for MDPs in accurately separating clusters in the MD space and remains robust against distortions. We further demonstrate the effectiveness of our technique through two use cases: (1) conducting cluster analysis of geospatial data and (2) interactively labeling MD clusters.
Safe Online Bid Optimization with Return on Investment and Budget Constraints
Venue: arXiv cs.LG 2025
Year: 2025

In online marketing, the advertisers aim to balance achieving high volumes and high profitability. The companies' business units address this tradeoff by maximizing the volumes while guaranteeing a minimum Return On Investment (ROI) level. Such a task can be naturally modeled as a combinatorial optimization problem subject to ROI and budget constraints that can be solved online. In this picture, the learner's uncertainty over the constraints' parameters plays a crucial role since the algorithms' exploration choices might lead to their violation during the entire learning process. Such violations represent a major obstacle to adopting online techniques in real-world applications. Thus, controlling the algorithms' exploration during learning is paramount to making humans trust online learning tools. This paper studies the nature of both optimization and learning problems. In particular, we show that the learning problem is inapproximable within any factor (unless P = NP) and provide a pseudo-polynomial-time algorithm to solve its discretized version. Subsequently, we prove that no online learning algorithm can violate the (ROI or budget) constraints a sublinear number of times during the learning process while guaranteeing a sublinear regret. We provide the $GCB$ algorithm that guarantees sublinear regret at the cost of a linear number of constraint violations and $GCB_{safe}$ that guarantees w.h.p. a constant upper bound on the number of constraint violations at the cost of a linear regret. Moreover, we designed $GCB_{safe}(ψ,ϕ)$, which guarantees both sublinear regret and safety w.h.p. at the cost of accepting tolerances $ψ$ and $ϕ$ in the satisfaction of the ROI and budget constraints, respectively. Finally, we provide experimental results to compare the regret and constraint violations of $GCB$, $GCB_{safe}$, and $GCB_{safe}(ψ,ϕ)$.
Near-Optimal Sparse Allreduce for Distributed Deep Learning
Venue: arXiv cs.DC 2025
Year: 2025

Communication overhead is one of the major obstacles to train large deep learning models at scale. Gradient sparsification is a promising technique to reduce the communication volume. However, it is very challenging to obtain real performance improvement because of (1) the difficulty of achieving an scalable and efficient sparse allreduce algorithm and (2) the sparsification overhead. This paper proposes O$k$-Top$k$, a scheme for distributed training with sparse gradients. O$k$-Top$k$ integrates a novel sparse allreduce algorithm (less than 6$k$ communication volume which is asymptotically optimal) with the decentralized parallel Stochastic Gradient Descent (SGD) optimizer, and its convergence is proved. To reduce the sparsification overhead, O$k$-Top$k$ efficiently selects the top-$k$ gradient values according to an estimated threshold. Evaluations are conducted on the Piz Daint supercomputer with neural network models from different deep learning domains. Empirical results show that O$k$-Top$k$ achieves similar model accuracy to dense allreduce. Compared with the optimized dense and the state-of-the-art sparse allreduces, O$k$-Top$k$ is more scalable and significantly improves training throughput (e.g., 3.29x-12.95x improvement for BERT on 256 GPUs).
Simple Calibration via Geodesic Kernels
Venue: arXiv cs.LG 2025
Year: 2025

Deep discriminative approaches, such as decision forests and deep neural networks, have recently found applications in many important real-world scenarios. However, deploying these learning algorithms in safety-critical applications raises concerns, particularly when it comes to ensuring calibration for both in-distribution and out-of-distribution regions. Many popular methods for in-distribution (ID) calibration, such as isotonic and Platt's sigmoidal regression, exhibit adequate ID calibration performance. However, these methods are not calibrated for the entire feature space, leading to overconfidence in the out-of-distribution (OOD) region. Existing OOD calibration methods generally exhibit poor ID calibration. In this paper, we jointly address the ID and OOD problems. We leveraged the fact that deep models learn to partition feature space into a union of polytopes, that is, flat-sided geometric objects. We introduce a geodesic distance to measure the distance between these polytopes and further distinguish samples within the same polytope using a Gaussian kernel. Our experiments on both tabular and vision benchmarks show that the proposed approaches, namely Kernel Density Forest (KDF) and Kernel Density Network (KDN), obtain well-calibrated posteriors for both ID and OOD samples, while mostly preserving the classification accuracy and extrapolating beyond the training data to handle OOD inputs appropriately.
The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective
Venue: arXiv cs.LG 2025
Year: 2025

As various post hoc explanation methods are increasingly being leveraged to explain complex models in high-stakes settings, it becomes critical to develop a deeper understanding of whether and when the explanations output by these methods disagree with each other, and how such disagreements are resolved in practice. However, there is little to no research that provides answers to these critical questions. In this work, we formalize and study the disagreement problem in explainable machine learning. More specifically, we define the notion of disagreement between explanations, analyze how often such disagreements occur in practice, and how practitioners resolve these disagreements. We first conduct interviews with data scientists to understand what constitutes disagreement between explanations generated by different methods for the same model prediction, and introduce a novel quantitative framework to formalize this understanding. We then leverage this framework to carry out a rigorous empirical analysis with four real-world datasets, six state-of-the-art post hoc explanation methods, and six different predictive models, to measure the extent of disagreement between the explanations generated by various popular explanation methods. In addition, we carry out an online user study with data scientists to understand how they resolve the aforementioned disagreements. Our results indicate that (1) state-of-the-art explanation methods often disagree in terms of the explanations they output, and (2) machine learning practitioners often employ ad hoc heuristics when resolving such disagreements. These findings suggest that practitioners may be relying on misleading explanations when making consequential decisions. They also underscore the importance of developing principled frameworks for effectively evaluating and comparing explanations output by various explanation techniques.
Ranking with Confidence for Large Scale Comparison Data
Venue: arXiv cs.LG 2025
Year: 2025

In this work, we leverage a generative data model considering comparison noise to develop a fast, precise, and informative ranking algorithm from pairwise comparisons that produces a measure of confidence on each comparison. The problem of ranking a large number of items from noisy and sparse pairwise comparison data arises in diverse applications, like ranking players in online games, document retrieval or ranking human perceptions. Although different algorithms are available, we need fast, large-scale algorithms whose accuracy degrades gracefully when the number of comparisons is too small. Fitting our proposed model entails solving a non-convex optimization problem, which we tightly approximate by a sum of quasi-convex functions and a regularization term. Resorting to an iterative reweighted minimization and the Primal-Dual Hybrid Gradient method, we obtain PD-Rank, achieving a Kendall tau 0.1 higher than all comparing methods, even for 10\% of wrong comparisons in simulated data matching our data model, and leading in accuracy if data is generated according to the Bradley-Terry model, in both cases faster by one order of magnitude, in seconds. In real data, PD-Rank requires less computational time to achieve the same Kendall tau than active learning methods.