OpenRaiser/Intern-Atlas huggingface dataset:
ELMES: An Automated Framework for Evaluating Large Language Models in Educational Scenarios
Venue: arXiv cs.CY 2025
Year: 2025

The emergence of Large Language Models (LLMs) presents transformative opportunities for education, generating numerous novel application scenarios. However, significant challenges remain: evaluation metrics vary substantially across different educational scenarios, while many emerging scenarios lack appropriate assessment metrics. Current benchmarks predominantly measure general intelligence rather than pedagogical capabilities. To address this gap, we introduce ELMES, an open-source automated evaluation framework specifically designed for assessing LLMs in educational settings. ELMES features a modular architecture that enables researchers to create dynamic, multi-agent dialogues through simple configuration files, facilitating flexible scenario design without requiring extensive programming expertise. The framework incorporates a hybrid evaluation engine that objectively quantifies traditionally subjective pedagogical metrics using an LLM-as-a-Judge methodology. We conduct systematic benchmarking of state-of-the-art LLMs across four critical educational scenarios: Knowledge Point Explanation, Guided Problem-Solving Teaching, Interdisciplinary Lesson Plan Generation, and Contextualized Question Generation, employing fine-grained metrics developed in collaboration with education specialists. Our results demonstrate distinct capability distributions among models, revealing context-specific strengths and limitations. ELMES provides educators and researchers with an accessible evaluation framework that significantly reduces adaptation barriers for diverse educational applications while advancing the practical implementation of LLMs in pedagogy. The framework is publicly available at \emph{https://github.com/sii-research/elmes.git}.
LLM-Assisted Cheating Detection in Korean Language via Keystrokes
Venue: arXiv cs.LG 2025
Year: 2025

This paper presents a keystroke-based framework for detecting LLM-assisted cheating in Korean, addressing key gaps in prior research regarding language coverage, cognitive context, and the granularity of LLM involvement. Our proposed dataset includes 69 participants who completed writing tasks under three conditions: Bona fide writing, paraphrasing ChatGPT responses, and transcribing ChatGPT responses. Each task spans six cognitive processes defined in Bloom's Taxonomy (remember, understand, apply, analyze, evaluate, and create). We extract interpretable temporal and rhythmic features and evaluate multiple classifiers under both Cognition-Aware and Cognition-Unaware settings. Temporal features perform well under Cognition-Aware evaluation scenarios, while rhythmic features generalize better under cross-cognition scenarios. Moreover, detecting bona fide and transcribed responses was easier than paraphrased ones for both the proposed models and human evaluators, with the models significantly outperforming the humans. Our findings affirm that keystroke dynamics facilitate reliable detection of LLM-assisted writing across varying cognitive demands and writing strategies, including paraphrasing and transcribing LLM-generated responses.
Axioms for Model Fidelity Evaluation
Venue: arXiv cs.CE 2025
Year: 2025

Digital engineering has transformed the design and development process. However, the utility of digital engineering is fundamentally dependent on the assumption that a simulation provides information consistent with reality. This relationship is described as model fidelity. Despite the widespread use of the term, existing definitions of model fidelity often lack formal rigor in practical application, which leaves ambiguity in how this similarity should be evaluated. This paper presents seven fundamental axioms to aid the development of future fidelity evaluation frameworks. An example of a ground vehicle model is used under an existing fidelity evaluation framework to observe the applicability of these axioms. In addition, these axioms are used as a reference point for considering future opportunities in future work related to model fidelity.
Reference-Guided Diffusion Inpainting For Multimodal Counterfactual Generation
Venue: arXiv cs.CV 2025
Year: 2025

Safety-critical applications, such as autonomous driving and medical image analysis, require extensive multimodal data for rigorous testing. Synthetic data methods are gaining prominence due to the cost and complexity of gathering real-world data, but they demand a high degree of realism and controllability to be useful. This work introduces two novel methods for synthetic data generation in autonomous driving and medical image analysis, namely MObI and AnydoorMed, respectively. MObI is a first-of-its-kind framework for Multimodal Object Inpainting that leverages a diffusion model to produce realistic and controllable object inpaintings across perceptual modalities, demonstrated simultaneously for camera and lidar. Given a single reference RGB image, MObI enables seamless object insertion into existing multimodal scenes at a specified 3D location, guided by a bounding box, while maintaining semantic consistency and multimodal coherence. Unlike traditional inpainting methods that rely solely on edit masks, this approach uses 3D bounding box conditioning to ensure accurate spatial positioning and realistic scaling. AnydoorMed extends this paradigm to the medical imaging domain, focusing on reference-guided inpainting for mammography scans. It leverages a diffusion-based model to inpaint anomalies with impressive detail preservation, maintaining the reference anomaly's structural integrity while semantically blending it with the surrounding tissue. Together, these methods demonstrate that foundation models for reference-guided inpainting in natural images can be readily adapted to diverse perceptual modalities, paving the way for the next generation of systems capable of constructing highly realistic, controllable and multimodal counterfactual scenarios.
Moravec's Paradox: Towards an Auditory Turing Test
Venue: arXiv cs.AI 2025
Year: 2025

This research work demonstrates that current AI systems fail catastrophically on auditory tasks that humans perform effortlessly. Drawing inspiration from Moravec's paradox (i.e., tasks simple for humans often prove difficult for machines, and vice versa), we introduce an auditory Turing test comprising 917 challenges across seven categories: overlapping speech, speech in noise, temporal distortion, spatial audio, coffee-shop noise, phone distortion, and perceptual illusions. Our evaluation of state-of-the-art audio models including GPT-4's audio capabilities and OpenAI's Whisper reveals a striking failure rate exceeding 93%, with even the best-performing model achieving only 6.9% accuracy on tasks that humans solved at 7.5 times higher success (52%). These results expose focusing failures in how AI systems process complex auditory scenes, particularly in selective attention, noise robustness, and contextual adaptation. Our benchmark not only quantifies the human-machine auditory gap but also provides insights into why these failures occur, suggesting that current architectures lack fundamental mechanisms for human-like auditory scene analysis. The traditional design of audio CAPTCHAs highlights common filters that humans evolved but machines fail to select in multimodal language models. This work establishes a diagnostic framework for measuring progress toward human-level machine listening and highlights the need for novel approaches integrating selective attention, physics-based audio understanding, and context-aware perception into multimodal AI systems.
FlowETL: An Autonomous Example-Driven Pipeline for Data Engineering
Venue: arXiv cs.SE 2025
Year: 2025

The Extract, Transform, Load (ETL) workflow is fundamental for populating and maintaining data warehouses and other data stores accessed by analysts for downstream tasks. A major shortcoming of modern ETL solutions is the extensive need for a human-in-the-loop, required to design and implement context-specific, and often non-generalisable transformations. While related work in the field of ETL automation shows promising progress, there is a lack of solutions capable of automatically designing and applying these transformations. We present FlowETL, a novel example-based autonomous ETL pipeline architecture designed to automatically standardise and prepare input datasets according to a concise, user-defined target dataset. FlowETL is an ecosystem of components which interact together to achieve the desired outcome. A Planning Engine uses a paired input-output datasets sample to construct a transformation plan, which is then applied by an ETL worker to the source dataset. Monitoring and logging provide observability throughout the entire pipeline. The results show promising generalisation capabilities across 14 datasets of various domains, file structures, and file sizes.
Countering the Forgetting of Novel Health Information with 'Social Boosting'
Venue: arXiv cs.SI 2025
Year: 2025

To mitigate the adverse effects of low-quality or false information, studies have shown the effectiveness of various intervention techniques through debunking or so-called pre-bunking. However, the effectiveness of such interventions can decay. Here, we investigate the role of the detailed social structure of the local villages within which the intervened individuals live, which provides opportunities for the targeted individuals to discuss and internalize new knowledge. We evaluated this with respect to a critically important topic, information about maternal and child health care, delivered via a 22-month in-home intervention. Specifically, we examined the effect of having friendship ties on the retention of knowledge interventions among targeted individuals in 110 isolated Honduran villages. We hypothesize that individuals who receive specific knowledge can internalize and consolidate this information by engaging in social interactions where, for instance, they have an opportunity to discuss it with others in the process. The opportunity to explain information to others (knowledge sharing) promotes deeper cognitive processing and elaborative encoding, which ultimately enhances memory retention. We found that well-connected individuals within a social network experience an enhanced effectiveness of knowledge interventions. These individuals may be more likely to internalize and retain the information and reinforce it in others, due to increased opportunities for social interaction where they teach others or learn from them, a mechanism we refer to as "social boosting". These findings underscore the role of social interactions in reinforcing health knowledge interventions over the long term. We believe these findings would be of interest to the health policy, the global health workforce, and healthcare professionals focusing on disadvantaged populations and UN missions on infodemics.
BAR Conjecture: the Feasibility of Inference Budget-Constrained LLM Services with Authenticity and Reasoning
Venue: arXiv cs.LG 2025
Year: 2025

When designing LLM services, practitioners care about three key properties: inference-time budget, factual authenticity, and reasoning capacity. However, our analysis shows that no model can simultaneously optimize for all three. We formally prove this trade-off and propose a principled framework named The BAR Theorem for LLM-application design.
The Construction of Near-optimal Universal Coding of Integers
Venue: arXiv cs.IT 2025
Year: 2025

Universal Coding of Integers (UCI) is suitable for discrete memoryless sources with unknown probability distributions and infinitely countable alphabet sizes. The UCI is a class of prefix codes, such that the ratio of the average codeword length to $\max\{1, H(P)\}$ is within a constant expansion factor $K_{\mathcal{C}}$ for any decreasing probability distribution $P$, where $H(P)$ is the entropy of $P$. For any UCI code $\mathcal{C}$, define \emph{the minimum expansion factor} $K_{\mathcal{C}}^{*}$ to represent the infimum of the set of extension factors of $\mathcal{C}$. Each $\mathcal{C}$ has a unique corresponding $K_{\mathcal{C}}^{*}$, and the smaller $K_{\mathcal{C}}^{*}$ is, the better the compression performance of $\mathcal{C}$ is. A class of UCI $\mathcal{C}$ (or family $\{\mathcal{C}_i\}_{i=1}^{\infty}$) achieving the smallest $K_{\mathcal{C}}^{*}$ is defined as the \emph{optimal UCI}. The best result currently is that the range of $C_{\mathcal{C}}^{*}$ for the optimal UCI is $2\leq C_{\mathcal{C}}^{*}\leq 2.5$. In this paper, we prove that there exists a class of near-optimal UCIs, called $ν$ code, to achieve $K_ν=2.0386$. This narrows the range of the minimum expansion factor for optimal UCI to $2\leq C_{\mathcal{C}}^{*}\leq 2.0386$. Another new class of UCI, called $Δδ$ code, is specifically constructed. We show that the $Δδ$ code and $ν$ code are currently optimal in terms of minimum expansion factor. In addition, we propose a new proof that shows the minimum expansion factor of the optimal UCI is lower bounded by $2$.
NaN-Propagation: A Novel Method for Sparsity Detection in Black-Box Computational Functions
Venue: arXiv cs.LG 2025
Year: 2025

When numerically evaluating a function's gradient, sparsity detection can enable substantial computational speedups through Jacobian coloring and compression. However, sparsity detection techniques for black-box functions are limited, and existing finite-difference-based methods suffer from false negatives due to coincidental zero gradients. These false negatives can silently corrupt gradient calculations, leading to difficult-to-diagnose errors. We introduce NaN-propagation, which exploits the universal contamination property of IEEE 754 Not-a-Number values to trace input-output dependencies through floating-point numerical computations. By systematically contaminating inputs with NaN and observing which outputs become NaN, the method reconstructs conservative sparsity patterns that eliminate a major source of false negatives. We demonstrate this approach on an aerospace wing weight model, achieving a 1.52x speedup while uncovering dozens of dependencies missed by conventional methods -- a significant practical improvement since gradient computation is often the bottleneck in optimization workflows. The technique leverages IEEE 754 compliance to work across programming languages and math libraries without requiring modifications to existing black-box codes. Furthermore, advanced strategies such as NaN payload encoding via direct bit manipulation enable faster-than-linear time complexity, yielding speed improvements over existing black-box sparsity detection methods. Practical algorithms are also proposed to mitigate challenges from branching code execution common in engineering applications.
Efficient DFT of Zadoff-Chu Sequences using lmFH Pattern
Venue: arXiv cs.IT 2025
Year: 2025

Having established that Zadoff-Chu (ZC) sequences are inherently linear micro-frequency hopping (lmFH) symbols, this paper first presents an intuitive and visual exposition of the computation of the DFT and IDFT of ZC sequences using the lmFH pattern. This yields interesting results. Subsequently, an alternative form for computing the cumulative sum of ZC sequences using the Generalized Quadratic Gauss Sum is introduced. Furthermore, building on the micro-frequency hopping (mFH) concept, this paper shows that the DFT of ZC sequences can be transformed into an lmFH symbol with frequency shift and phase offset. Therefore, the DFT of ZC sequences can be computed via cumulative frequency points, similar to the computation of normal mFH symbols.
Quadratic Programming-Based Posture Manipulation and Thrust-vectoring for Agile Dynamic Walking on Narrow Pathways
Venue: arXiv cs.RO 2025
Year: 2025

There has been significant advancement in legged robot's agility where they can show impressive acrobatic maneuvers, such as parkour. These maneuvers rely heavily on posture manipulation. To expand the stability and locomotion plasticity, we use the multi-modal ability in our legged-aerial platform, the Husky Beta, to perform thruster-assisted walking. This robot has thrusters on each of its sagittal knee joints which can be used to stabilize its frontal dynamic as it walks. In this work, we perform a simulation study of quadruped narrow-path walking with Husky $β$, where the robot will utilize its thrusters to stably walk on a narrow path. The controller is designed based on a centroidal dynamics model with thruster and foot ground contact forces as inputs. These inputs are regulated using a QP solver to be used in a model predictive control framework. In addition to narrow-path walking, we also perform a lateral push-recovery simulation to study how the thrusters can be used to stabilize the frontal dynamics.
Kernel-FFI: Transparent Foreign Function Interfaces for Interactive Notebooks
Venue: arXiv cs.PL 2025
Year: 2025

Foreign Function Interfaces (FFIs) are essential for enabling interoperability between programming languages, yet existing FFI solutions are ill-suited for the dynamic, interactive workflows prevalent in modern notebook environments such as Jupyter. Current approaches require extensive manual configuration, introduce significant boilerplate, and often lack support for recursive calls and object-oriented programming (OOP) constructs-features critical for productive, multi-language development. We present Kernel-FFI, a transparent, language-agnostic framework that enables seamless cross-language function calls and object manipulation within interactive notebooks. Kernel-FFI employs source-level transformation to automatically rewrite cross-language invocations, eliminating the need for manual bindings or boilerplate. Kernel-FFI provides robust support for OOP by enabling foreign object referencing and automatic resource management across language boundaries. Furthermore, to address the blocking nature of Jupyter kernels and support recursive and asynchronous foreign calls, we introduce a novel side-channel communication mechanism. Our tool will be open-sourced and available at https://codepod.io/docs/kernel-ffi
Enabling Few-Shot Alzheimer's Disease Diagnosis on Biomarker Data with Tabular LLMs
Venue: arXiv cs.CL 2025
Year: 2025

Early and accurate diagnosis of Alzheimer's disease (AD), a complex neurodegenerative disorder, requires analysis of heterogeneous biomarkers (e.g., neuroimaging, genetic risk factors, cognitive tests, and cerebrospinal fluid proteins) typically represented in a tabular format. With flexible few-shot reasoning, multimodal integration, and natural-language-based interpretability, large language models (LLMs) offer unprecedented opportunities for prediction with structured biomedical data. We propose a novel framework called TAP-GPT, Tabular Alzheimer's Prediction GPT, that adapts TableGPT2, a multimodal tabular-specialized LLM originally developed for business intelligence tasks, for AD diagnosis using structured biomarker data with small sample sizes. Our approach constructs few-shot tabular prompts using in-context learning examples from structured biomedical data and finetunes TableGPT2 using the parameter-efficient qLoRA adaption for a clinical binary classification task of AD or cognitively normal (CN). The TAP-GPT framework harnesses the powerful tabular understanding ability of TableGPT2 and the encoded prior knowledge of LLMs to outperform more advanced general-purpose LLMs and a tabular foundation model (TFM) developed for prediction tasks. To our knowledge, this is the first application of LLMs to the prediction task using tabular biomarker data, paving the way for future LLM-driven multi-agent frameworks in biomedical informatics.
Fine-Grained Privacy Extraction from Retrieval-Augmented Generation Systems via Knowledge Asymmetry Exploitation
Venue: arXiv cs.CR 2025
Year: 2025

Retrieval-augmented generation (RAG) systems enhance large language models (LLMs) by integrating external knowledge bases, but this advancement introduces significant privacy risks. Existing privacy attacks on RAG systems can trigger data leakage but often fail to accurately isolate knowledge-base-derived sentences within mixed responses. They also lack robustness when applied across multiple domains. This paper addresses these challenges by presenting a novel black-box attack framework that exploits knowledge asymmetry between RAG and standard LLMs to achieve fine-grained privacy extraction across heterogeneous knowledge landscapes. We propose a chain-of-thought reasoning strategy that creates adaptive prompts to steer RAG systems away from sensitive content. Specifically, we first decompose adversarial queries to maximize information disparity and then apply a semantic relationship scoring to resolve lexical and syntactic ambiguities. We finally train a neural network on these feature scores to precisely identify sentences containing private information. Unlike prior work, our framework generalizes to unseen domains through iterative refinement without pre-defined knowledge. Experimental results show that we achieve over 91% privacy extraction rate in single-domain and 83% in multi-domain scenarios, reducing sensitive sentence exposure by over 65% in case studies. This work bridges the gap between attack and defense in RAG systems, enabling precise extraction of private information while providing a foundation for adaptive mitigation.
Secure Integrated Sensing and Communication Networks: Stochastic Performance Analysis
Venue: arXiv cs.IT 2025
Year: 2025

This paper analyzes the stochastic security performance of a multiple-input multiple-output (MIMO) integrated sensing and communication (ISAC) system in a downlink scenario. A base station (BS) transmits a multi-functional signal to simultaneously communicate with a user, sense a target's angular location, and counteract eavesdropping threats. The attack model considers a passive single-antenna communication eavesdropper intercepting communication data, as well as a multi-antenna sensing eavesdropper attempting to infer the target's location. We also consider a malicious target scenario where the target plays the role of the communication eavesdropper. The BS-user and BS-eavesdroppers channels follow Rayleigh fading, while the target's azimuth angle is uniformly distributed. To evaluate the performance in this random network, we derive the ergodic secrecy rate (ESR) and the ergodic Cramer-Rao lower bound (CRB), for target localization, at both the BS and the sensing eavesdropper. This involves computing the probability density functions (PDFs) of the signal-to-noise ratio (SNR) and CRB, leveraging the central limit theorem for tractability. We characterize the boundary of the CRB-secrecy rate region, and interpret the performance tradeoffs between communication and sensing while guaranteeing a level of security and privacy in the random ISAC networks.