OpenRaiser/Intern-Atlas huggingface dataset:
Improved Sublinear-time Moment Estimation using Weighted Sampling
Venue: arXiv cs.DS 2025
Year: 2025

In this work we study the {\it moment estimation} problem using weighted sampling. Given sample access to a set $A$ with $n$ weighted elements, and a parameter $t>0$, we estimate the $t$-th moment of $A$ given as $S_t=\sum_{a\in A} w(a)^t$. For t=1, this is the sum estimation problem. The moment estimation problem along with a number of its variants have been extensively studied in streaming, sublinear and distributed communication models. Despite being well studied, we don't yet have a complete understanding of the sample complexity of the moment estimation problem in the sublinear model and in this work, we make progress on this front. On the algorithmic side, our upper bounds match the known upper bounds for the problem for $t>1$. To the best of our knowledge, no sublinear algorithms were known for this problem for $0<t<1$. We design a sublinear algorithm for this problem for $t>1/2$ and show that no sublinear algorithms exist for $t\leq 1/2$. We prove a $Ω(\frac{n^{1-1/t}\ln 1/δ}{ε^2})$ lower bound for moment estimation for $t>1$, and show optimal sample complexity bound $Θ(\frac{n^{1-1/t}\ln 1/δ}{ε^2})$ for moment estimation for $t\geq 2$. Hence, we obtain a complete understanding of the sample complexity for moment estimation using proportional sampling for $t\geq 2$. We also study the moment estimation problem in the beyond worst-case analysis paradigm and identify a new {\it moment-density} parameter of the input that characterizes the sample complexity of the problem using proportional sampling and derive tight sample complexity bounds with respect to that parameter. We also study the moment estimation problem in the hybrid sampling framework in which one is given additional access to a uniform sampling oracle and show that hybrid sampling framework does not provide any additional gain over the proportional sampling oracle in the worst case.
Stepwise Informativeness Search for Efficient and Effective LLM Reasoning
Venue: arXiv cs.CL 2025
Year: 2025

Advances in Large Language Models (LLMs) have significantly improved multi-step reasoning through generating free-text rationales. However, recent studies show that LLMs tend to lose focus over the middle of long contexts. This raises concerns that as reasoning progresses, LLMs may overlook information in earlier steps when decoding subsequent steps, leading to generate unreliable and redundant rationales. To address this, we propose guiding LLMs to generate more accurate and concise step-by-step rationales by (1) proactively referencing information from underutilized prior steps, and (2) minimizing redundant information between new and existing steps. We introduce stepwise informativeness search, an inference-time tree search framework incorporating two selection heuristics: grounding-guided selection which prioritizes steps paying higher attention over underutilized steps; and novelty-guided selection which encourages steps with novel conclusions. During rationale generation, we use a self-grounding strategy that prompts LLMs to explicitly reference relevant prior steps to provide premises before deduction at each step. Experimental results on four reasoning datasets demonstrate that our approach improves reasoning accuracy by generating higher-quality rationales with reduced errors and redundancy.
Efficiently Solving Discounted MDPs with Predictions on Transition Matrices
Venue: arXiv cs.LG 2025
Year: 2025

We study infinite-horizon Discounted Markov Decision Processes (DMDPs) under a generative model. Motivated by the Algorithm with Advice framework Mitzenmacher and Vassilvitskii 2022, we propose a novel framework to investigate how a prediction on the transition matrix can enhance the sample efficiency in solving DMDPs and improve sample complexity bounds. We focus on the DMDPs with $N$ state-action pairs and discounted factor $γ$. Firstly, we provide an impossibility result that, without prior knowledge of the prediction accuracy, no sampling policy can compute an $ε$-optimal policy with a sample complexity bound better than $\tilde{O}((1-γ)^{-3} Nε^{-2})$, which matches the state-of-the-art minimax sample complexity bound with no prediction. In complement, we propose an algorithm based on minimax optimization techniques that leverages the prediction on the transition matrix. Our algorithm achieves a sample complexity bound depending on the prediction error, and the bound is uniformly better than $\tilde{O}((1-γ)^{-4} N ε^{-2})$, the previous best result derived from convex optimization methods. These theoretical findings are further supported by our numerical experiments.
Effects of a Co-Regulation Model for MR Teacher Training: HRV and Self-Compassion as Indicators of Emotion Regulation
Venue: arXiv cs.HC 2025
Year: 2025

Teachers play a pivotal role in fostering students' emotional and cognitive development. Teachers need to regulate their emotions in order to co-regulate students. Here using a unique mixed method approach, we investigate the relationship between self-compassion, treating oneself with compassion, and physiological stress responses among pre-service teachers. Heart rate variability (HRV) was measured during a mixed reality (MR) teacher training scenario environment designed to simulate socio-emotional conflict in class. Recorded interviews that followed the MR-training were analyzed for observed self-compassion. Findings suggest that less emotional stress during the MR-training correlates with higher levels of self-compassion during the interview. MR-trainings and self-compassion may be valuable tools to train teacher emotion regulation and well-being.
The Role of Background Information in Reducing Object Hallucination in Vision-Language Models: Insights from Cutoff API Prompting
Venue: arXiv cs.CV 2025
Year: 2025

Vision-Language Models (VLMs) occasionally generate outputs that contradict input images, constraining their reliability in real-world applications. While visual prompting is reported to suppress hallucinations by augmenting prompts with relevant area inside an image, the effectiveness in terms of the area remains uncertain. This study analyzes success and failure cases of Attention-driven visual prompting in object hallucination, revealing that preserving background context is crucial for mitigating object hallucination.
Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning
Venue: arXiv cs.CL 2025
Year: 2025

In-context learning (ICL) can significantly enhance the complex reasoning capabilities of large language models (LLMs), with the key lying in the selection and ordering of demonstration examples. Previous methods typically relied on simple features to measure the relevance between examples. We argue that these features are not sufficient to reflect the intrinsic connections between examples. In this study, we propose a curriculum ICL strategy guided by problem-solving logic. We select demonstration examples by analyzing the problem-solving logic and order them based on curriculum learning. Specifically, we constructed a problem-solving logic instruction set based on the BREAK dataset and fine-tuned a language model to analyze the problem-solving logic of examples. Subsequently, we selected appropriate demonstration examples based on problem-solving logic and assessed their difficulty according to the number of problem-solving steps. In accordance with the principles of curriculum learning, we ordered the examples from easy to hard to serve as contextual prompts. Experimental results on multiple benchmarks indicate that our method outperforms previous ICL approaches in terms of performance and efficiency, effectively enhancing the complex reasoning capabilities of LLMs. Our project will be released at https://github.com/maxuetao/CurriculumICL
Enabling Seamless Creation of Annotated Spaces: Enhancing Learning in VR Environments
Venue: arXiv cs.HC 2025
Year: 2025

We present an approach to evaluate the efficacy of annotations in augmenting learning environments in the context of Virtual Reality. Our study extends previous work highlighting the benefits of learning based in virtual reality and introduces a method to facilitate asynchronous collaboration between educators and students. These two distinct perspectives fulfill special roles: educators aim to convey information, which learners should get familiarized. Educators are empowered to annotate static scenes on large touchscreens to supplement information. Subsequently, learners explore those annotated scenes in virtual reality. To assess the comparative ease and usability of creating text and pen annotations, we conducted a user study with 24 participants, which assumed both roles of learners and teachers. Educators annotated static courses using provided textbook excerpts, interfacing through an 86-inch touchscreen. Learners navigated pre-designed educational courses in virtual reality to evaluate the practicality of annotations. The utility of annotations in virtual reality garnered high ratings. Users encountered issues with the touch interface implementation and rated it with a low intuitivity. Despite this, our study underscores the significant benefits of annotations, particularly for learners. This research offers valuable insights into annotation-enriched learning, emphasizing its potential to enhance students' information retention and comprehension.
Mixup Model Merge: Enhancing Model Merging Performance through Randomized Linear Interpolation
Venue: arXiv cs.CL 2025
Year: 2025

Model merging aims to integrate multiple task-specific models into a unified model that inherits the capabilities of the task-specific models, without additional training. Existing model merging methods often lack consideration of the varying contribution ratios of different task-specific models to the final merged model. In this paper, we propose Mixup Model Merge (M3), a simple yet effective method inspired by the randomized linear interpolation strategy from the Mixup data augmentation technique. M3 performs randomized linear interpolation in parameter space between two task-specific LLMs, where interpolation coefficients are sampled from a Beta distribution to explore diverse contribution ratios. This controllable randomness allows M3 to outperform standard equal-ratio merging by discovering better contribution ratio combinations. Extensive experiments show that M3 significantly (1) improves merged LLM performance across tasks, (2) enhances out-of-distribution and adversarial robustness, (3) outperforms the positive effects of the sparsification method DARE on model merging and can be further combined with DARE to achieve superior results, and (4) balances exploration efficiency and diversity in contribution ratios by tuning the Beta distribution's shape parameters. The code is provided in the supplementary materials.
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models
Venue: arXiv cs.CL 2025
Year: 2025

Large language models (LLMs) exhibit excellent performance in various tasks. However, the memory requirements of LLMs present a great challenge when deploying on memory-limited devices, even for quantized LLMs. This paper introduces a framework to compress LLM after quantization further, achieving about 2.2x compression ratio. A compression-aware quantization is first proposed to enhance model weight compressibility by re-scaling the model parameters before quantization, followed by a pruning method to improve further. Upon this, we notice that decompression can be a bottleneck during practical scenarios. We then give a detailed analysis of the trade-off between memory usage and latency brought by the proposed method. A speed-adaptive method is proposed to overcome it. The experimental results show inference with the compressed model can achieve a 40% reduction in memory size with negligible loss in accuracy and inference speed.
PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing System
Venue: arXiv cs.AR 2025
Year: 2025

Large language models (LLMs) are widely used for natural language understanding and text generation. An LLM model relies on a time-consuming step called LLM decoding to generate output tokens. Several prior works focus on improving the performance of LLM decoding using parallelism techniques, such as batching and speculative decoding. State-of-the-art LLM decoding has both compute-bound and memory-bound kernels. Some prior works statically identify and map these different kernels to a heterogeneous architecture consisting of both processing-in-memory (PIM) units and computation-centric accelerators. We observe that characteristics of LLM decoding kernels (e.g., whether or not a kernel is memory-bound) can change dynamically due to parameter changes to meet user and/or system demands, making (1) static kernel mapping to PIM units and computation-centric accelerators suboptimal, and (2) one-size-fits-all approach of designing PIM units inefficient due to a large degree of heterogeneity even in memory-bound kernels. In this paper, we aim to accelerate LLM decoding while considering the dynamically changing characteristics of the kernels involved. We propose PAPI (PArallel Decoding with PIM), a PIM-enabled heterogeneous architecture that exploits dynamic scheduling of compute-bound or memory-bound kernels to suitable hardware units. PAPI has two key mechanisms: (1) online kernel characterization to dynamically schedule kernels to the most suitable hardware units at runtime and (2) a PIM-enabled heterogeneous computing system that harmoniously orchestrates both computation-centric processing units and hybrid PIM units with different computing capabilities. Our experimental results on three broadly-used LLMs show that PAPI achieves 1.8$\times$ and 11.1$\times$ speedups over a state-of-the-art heterogeneous LLM accelerator and a state-of-the-art PIM-only LLM accelerator, respectively.
Decoding for Punctured Convolutional and Turbo Codes: A Deep Learning Solution for Protocols Compliance
Venue: arXiv cs.LG 2025
Year: 2025

Neural network-based decoding methods show promise in enhancing error correction performance but face challenges with punctured codes. In particular, existing methods struggle to adapt to variable code rates or meet protocol compatibility requirements. This paper proposes a unified long short-term memory (LSTM)-based neural decoder for punctured convolutional and Turbo codes to address these challenges. The key component of the proposed LSTM-based neural decoder is puncturing-aware embedding, which integrates puncturing patterns directly into the neural network to enable seamless adaptation to different code rates. Moreover, a balanced bit error rate training strategy is designed to ensure the decoder's robustness across various code lengths, rates, and channels. In this way, the protocol compatibility requirement can be realized. Extensive simulations in both additive white Gaussian noise (AWGN) and Rayleigh fading channels demonstrate that the proposed neural decoder outperforms conventional decoding techniques, offering significant improvements in decoding accuracy and robustness.
On Neural BRDFs: A Thorough Comparison of State-of-the-Art Approaches
Venue: arXiv cs.CV 2025
Year: 2025

The bidirectional reflectance distribution function (BRDF) is an essential tool to capture the complex interaction of light and matter. Recently, several works have employed neural methods for BRDF modeling, following various strategies, ranging from utilizing existing parametric models to purely neural parametrizations. While all methods yield impressive results, a comprehensive comparison of the different approaches is missing in the literature. In this work, we present a thorough evaluation of several approaches, including results for qualitative and quantitative reconstruction quality and an analysis of reciprocity and energy conservation. Moreover, we propose two extensions that can be added to existing approaches: A novel additive combination strategy for neural BRDFs that split the reflectance into a diffuse and a specular part, and an input mapping that ensures reciprocity exactly by construction, while previous approaches only ensure it by soft constraints.
FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detection
Venue: arXiv cs.CV 2025
Year: 2025

Camera-based multi-view 3D detection is crucial for autonomous driving. PETR and its variants (PETRs) excel in benchmarks but face deployment challenges due to high computational cost and memory footprint. Quantization is an effective technique for compressing deep neural networks by reducing the bit width of weights and activations. However, directly applying existing quantization methods to PETRs leads to severe accuracy degradation. This issue primarily arises from two key challenges: (1) significant magnitude disparity between multi-modal features-specifically, image features and camera-ray positional embeddings (PE), and (2) the inefficiency and approximation error of quantizing non-linear operators, which commonly rely on hardware-unfriendly computations. In this paper, we propose FQ-PETR, a fully quantized framework for PETRs, featuring three key innovations: (1) Quantization-Friendly LiDAR-ray Position Embedding (QFPE): Replacing multi-point sampling with LiDAR-prior-guided single-point sampling and anchor-based embedding eliminates problematic non-linearities (e.g., inverse-sigmoid) and aligns PE scale with image features, preserving accuracy. (2) Dual-Lookup Table (DULUT): This algorithm approximates complex non-linear functions using two cascaded linear LUTs, achieving high fidelity with minimal entries and no specialized hardware. (3) Quantization After Numerical Stabilization (QANS): Performing quantization after softmax numerical stabilization mitigates attention distortion from large inputs. On PETRs (e.g. PETR, StreamPETR, PETRv2, MV2d), FQ-PETR under W8A8 achieves near-floating-point accuracy (1% degradation) while reducing latency by up to 75%, significantly outperforming existing PTQ and QAT baselines.
Programmers Aren't Obsolete Yet: A Syllabus for Teaching CS Students to Responsibly Use Large Language Models for Code Generation
Venue: arXiv cs.SE 2025
Year: 2025

Large Language Models (LLMs) have emerged as powerful tools for automating code generation, offering immense potential to enhance programmer productivity. However, their non-deterministic nature and reliance on user input necessitate a robust understanding of programming fundamentals to ensure their responsible and effective use. In this paper, we argue that foundational computing skills remain crucial in the age of LLMs. We propose a syllabus focused on equipping computer science students to responsibly embrace LLMs as performance enhancement tools. This work contributes to the discussion on the why, when, and how of integrating LLMs into computing education, aiming to better prepare programmers to leverage these tools without compromising foundational software development principles.
Promoting Gender Equality in Competitive Programming: Strategies and Impacts of Affirmative Actions in Programming Marathons in Brazil
Venue: arXiv cs.CY 2025
Year: 2025

In the context of Computing, competitive programming is a relevant area that aims to have students, usually in teams, solve programming challenges, developing skills and competencies in the field. However, female participation remains significantly low and notably distant compared to male participation, even with proven intellectual equity between genders. This research aims to present strategies used to improve female participation in Programming Marathons in Brasil. The developed research is documentary, applied, and exploratory, with actions that generate results for female participation, with affirmative and inclusion actions, an important step towards gender equity in competitive programming.
Bridging vision language model (VLM) evaluation gaps with a framework for scalable and cost-effective benchmark generation
Venue: arXiv cs.CV 2025
Year: 2025

Reliable evaluation of AI models is critical for scientific progress and practical application. While existing VLM benchmarks provide general insights into model capabilities, their heterogeneous designs and limited focus on a few imaging domains pose significant challenges for both cross-domain performance comparison and targeted domain-specific evaluation. To address this, we propose three key contributions: (1) a framework for the resource-efficient creation of domain-specific VLM benchmarks enabled by task augmentation for creating multiple diverse tasks from a single existing task, (2) the release of new VLM benchmarks for seven domains, created according to the same homogeneous protocol and including 162,946 thoroughly human-validated answers, and (3) an extensive benchmarking of 22 state-of-the-art VLMs on a total of 37,171 tasks, revealing performance variances across domains and tasks, thereby supporting the need for tailored VLM benchmarks. Adoption of our methodology will pave the way for the resource-efficient domain-specific selection of models and guide future research efforts toward addressing core open questions.
Adaptive Expansion for Hypergraph Learning
Venue: arXiv cs.SI 2025
Year: 2025

Hypergraph, with its powerful ability to capture higher-order relationships, has gained significant attention recently. Consequently, many hypergraph representation learning methods have emerged to model the complex relationships among hypergraphs. In general, these methods leverage classic expansion methods to convert hypergraphs into weighted or bipartite graphs, and further employ message passing mechanisms to model the complex structures within hypergraphs. However, classical expansion methods are designed in straightforward manners with fixed edge weights, resulting in information loss or redundancy. In light of this, we design a novel clique expansion-based Adaptive Expansion method called AdE to adaptively expand hypergraphs into weighted graphs that preserve the higher-order structure information. Specifically, we introduce a novel Global Simulation Network to select two representative nodes for adaptively symbolizing each hyperedge and connect the rest of the nodes within the same hyperedge to the corresponding selected nodes. Afterward, we design a distance-aware kernel function, dynamically adjusting edge weights to ensure similar nodes within a hyperedge are connected with larger weights. Extensive theoretical justifications and empirical experiments over seven benchmark hypergraph datasets demonstrate that AdE has excellent rationality, generalization, and effectiveness compared to classic expansion models.
DReSD: Dense Retrieval for Speculative Decoding
Venue: arXiv cs.CL 2025
Year: 2025

Speculative decoding (SD) accelerates Large Language Model (LLM) generation by using an efficient draft model to propose the next few tokens, which are verified by the LLM in a single forward call, reducing latency while preserving its outputs. We focus on retrieval-based SD where the draft model retrieves the next tokens from a non-parametric datastore. Sparse retrieval (REST), which operates on the surface form of strings, is currently the dominant paradigm due to its simplicity and scalability. However, its effectiveness is limited due to the usage of short contexts and exact string matching. Instead, we introduce Dense Retrieval for Speculative Decoding (DReSD), a novel framework that uses approximate nearest neighbour search with contextualised token embeddings to retrieve the most semantically relevant token sequences for SD. Extensive experiments show that DReSD achieves (on average) 87% higher acceptance rates, 65% longer accepted tokens and 19% faster generation speeds compared to sparse retrieval (REST).