OpenRaiser/Intern-Atlas huggingface dataset:
Validity Verification of the New TOEFL Writing Task Based on Classical Test Theory
Venue: arXiv cs.CY 2025
Year: 2025

The TOEFL iBT has introduced the Academic Discussion Task (ADT) to assess test-takers' ability to engage in academic discourse, reflecting the growing emphasis on interactive communication skills in higher education. However, research on the ADT's validity and fairness particularly for culturally and linguistically diverse groups, such as Chinese students, remains limited. This study addresses this gap by employing Classical Test Theory (CTT) to evaluate the psychometric properties of the ADT among Chinese university students. This study finds a robust correlation between the ADT and the CET-6 writing and translation subscores. In addition, there is a high level of expert agreement regarding the construct validity evidence and the appropriateness of the scoring rubric. Furthermore, the results indicate that gender differences in validity indices are minimal. Taken together, these results suggest that the ADT is a valid measure for Chinese test-takers without gender discrimination. However, it is recommended that the cultural sensitivity of the scoring rubric be further refined and that the CET-6 subscores for writing be retained for predictive purposes, in order to better accommodate the needs of diverse test-taker populations. By addressing these issues, this study contributes to the broader discourse on fairness and validity in high-stakes language assessments.
Unsupervised Instance Segmentation with Superpixels
Venue: arXiv cs.CV 2025
Year: 2025

Instance segmentation is essential for numerous computer vision applications, including robotics, human-computer interaction, and autonomous driving. Currently, popular models bring impressive performance in instance segmentation by training with a large number of human annotations, which are costly to collect. For this reason, we present a new framework that efficiently and effectively segments objects without the need for human annotations. Firstly, a MultiCut algorithm is applied to self-supervised features for coarse mask segmentation. Then, a mask filter is employed to obtain high-quality coarse masks. To train the segmentation network, we compute a novel superpixel-guided mask loss, comprising hard loss and soft loss, with high-quality coarse masks and superpixels segmented from low-level image features. Lastly, a self-training process with a new adaptive loss is proposed to improve the quality of predicted masks. We conduct experiments on public datasets in instance segmentation and object detection to demonstrate the effectiveness of the proposed framework. The results show that the proposed framework outperforms previous state-of-the-art methods.
Human-LLM Synergy in Context-Aware Adaptive Architecture for Scalable Drone Swarm Operation
Venue: arXiv cs.RO 2025
Year: 2025

The deployment of autonomous drone swarms in disaster response missions necessitates the development of flexible, scalable, and robust coordination systems. Traditional fixed architectures struggle to cope with dynamic and unpredictable environments, leading to inefficiencies in energy consumption and connectivity. This paper addresses this gap by proposing an adaptive architecture for drone swarms, leveraging a Large Language Model to dynamically select the optimal architecture as centralized, hierarchical, or holonic based on real time mission parameters such as task complexity, swarm size, and communication stability. Our system addresses the challenges of scalability, adaptability, and robustness,ensuring efficient energy consumption and maintaining connectivity under varying conditions. Extensive simulations demonstrate that our adaptive architecture outperforms traditional static models in terms of scalability, energy efficiency, and connectivity. These results highlight the potential of our approach to provide a scalable, adaptable, and resilient solution for real world disaster response scenarios.
Prototyping an AI-powered Tool for Energy Efficiency in New Zealand Homes
Venue: arXiv cs.CY 2025
Year: 2025

Residential buildings contribute significantly to energy use, health outcomes, and carbon emissions. In New Zealand, housing quality has historically been poor, with inadequate insulation and inefficient heating contributing to widespread energy hardship. Recent reforms, including the Warmer Kiwi Homes program, Healthy Homes Standards, and H1 Building Code upgrades, have delivered health and comfort improvements, yet challenges persist. Many retrofits remain partial, data on household performance are limited, and decision-making support for homeowners is fragmented. This study presents the design and evaluation of an AI-powered decision-support tool for residential energy efficiency in New Zealand. The prototype, developed using Python and Streamlit, integrates data ingestion, anomaly detection, baseline modeling, and scenario simulation (e.g., LED retrofits, insulation upgrades) into a modular dashboard. Fifteen domain experts, including building scientists, consultants, and policy practitioners, tested the tool through semi-structured interviews. Results show strong usability (M = 4.3), high value of scenario outputs (M = 4.5), and positive perceptions of its potential to complement subsidy programs and regulatory frameworks. The tool demonstrates how AI can translate national policies into personalized, household-level guidance, bridging the gap between funding, standards, and practical decision-making. Its significance lies in offering a replicable framework for reducing energy hardship, improving health outcomes, and supporting climate goals. Future development should focus on carbon metrics, tariff modeling, integration with national datasets, and longitudinal trials to assess real-world adoption.
Influence of Stakeholder Involvement in M&E on the Performance of Donor-Funded Projects in Informal Settlements in Kisumu Central Sub-County, Kisumu County, Kenya
Venue: arXiv cs.CY 2025
Year: 2025

Stakeholder engagement and participatory approaches influence the effectiveness of donor-funded projects. Participatory Monitoring and Evaluation (PM&E) methodologies ensure that local communities play an active role in decision-making, leading to more sustainable outcomes. Given the complex socio-political landscape of Kisumu Central Sub-County, there is a critical need for inclusive and context-responsive project monitoring strategies. Initiatives that have integrated local leaders, youth groups, and women-led organizations into their M&E processes tend to achieve stronger community buy-in, improved continuity, and more impactful outcomes. The current study explored the influence of stakeholder involvement in M&E on the performance of donor-funded projects in informal settlements in Kisumu Central Sub-County, Kenya. The study was guided by Stakeholder Engagement Theory. The study used a Convergent Parallel design with a sample size of 364 respondents computed using Yamanes' Sampling formula, drawn from 27 donor-funded projects in Obunga and Nyalenda informal Settlements. Purposive sampling was used for project managers, project M&E staff, and community members served, while community members were selected using stratified random sampling. The study findings revealed that there were regular opportunities for stakeholder interaction in the projects (x =4.05, SD 1.08), stakeholders contributed to the development of the organization/project (x= 3.79, SD=.940), although stakeholders' perspectives and opinions were not diligently incorporated into programming (x=2.06, SD=.879) as anticipated. The study, therefore, concluded that stakeholder involvement in M&E influenced the performance of donor-funded projects. The study recommended that the project managers in donor-funded projects need to enhance stakeholder involvement for project ownership and sustainability.
Privacy Preservation and Identity Tracing Prevention in AI-Driven Eye Tracking for Interactive Learning Environments
Venue: arXiv cs.CR 2025
Year: 2025

Eye-tracking technology can aid in understanding neurodevelopmental disorders and tracing a person's identity. However, this technology poses a significant risk to privacy, as it captures sensitive information about individuals and increases the likelihood that data can be traced back to them. This paper proposes a human-centered framework designed to prevent identity backtracking while preserving the pedagogical benefits of AI-powered eye tracking in interactive learning environments. We explore how real-time data anonymization, ethical design principles, and regulatory compliance (such as GDPR) can be integrated to build trust and transparency. We first demonstrate the potential for backtracking student IDs and diagnoses in various scenarios using serious game-based eye-tracking data. We then provide a two-stage privacy-preserving framework that prevents participants from being tracked while still enabling diagnostic classification. The first phase covers four scenarios: I) Predicting disorder diagnoses based on different game levels. II) Predicting student IDs based on different game levels. III) Predicting student IDs based on randomized data. IV) Utilizing K-Means for out-of-sample data. In the second phase, we present a two-stage framework that preserves privacy. We also employ Federated Learning (FL) across multiple clients, incorporating a secure identity management system with dummy IDs and administrator-only access controls. In the first phase, the proposed framework achieved 99.3% accuracy for scenario 1, 63% accuracy for scenario 2, and 99.7% accuracy for scenario 3, successfully identifying and assigning a new student ID in scenario 4. In phase 2, we effectively prevented backtracking and established a secure identity management system with dummy IDs and administrator-only access controls, achieving an overall accuracy of 99.40%.
Evaluating Magic Leap 2 Tool Tracking for AR Sensor Guidance in Industrial Inspections
Venue: arXiv cs.RO 2025
Year: 2025

Rigorous evaluation of commercial Augmented Reality (AR) hardware is crucial, yet public benchmarks for tool tracking on modern Head-Mounted Displays (HMDs) are limited. This paper addresses this gap by systematically assessing the Magic Leap 2 (ML2) controllers tracking performance. Using a robotic arm for repeatable motion (EN ISO 9283) and an optical tracking system as ground truth, our protocol evaluates static and dynamic performance under various conditions, including realistic paths from a hydrogen leak inspection use case. The results provide a quantitative baseline of the ML2 controller's accuracy and repeatability and present a robust, transferable evaluation methodology. The findings provide a basis to assess the controllers suitability for the inspection use case and similar industrial sensor-based AR guidance tasks.
Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate
Venue: arXiv cs.CL 2025
Year: 2025

While multi-agent debate has been proposed as a promising strategy for improving AI reasoning ability, we find that debate can sometimes be harmful rather than helpful. Prior work has primarily focused on debates within homogeneous groups of agents, whereas we explore how diversity in model capabilities influences the dynamics and outcomes of multi-agent interactions. Through a series of experiments, we demonstrate that debate can lead to a decrease in accuracy over time - even in settings where stronger (i.e., more capable) models outnumber their weaker counterparts. Our analysis reveals that models frequently shift from correct to incorrect answers in response to peer reasoning, favoring agreement over challenging flawed reasoning. We perform additional experiments investigating various potential contributing factors to these harmful shifts - including sycophancy, social conformity, and model and task type. These results highlight important failure modes in the exchange of reasons during multi-agent debate, suggesting that naive applications of debate may cause performance degradation when agents are neither incentivised nor adequately equipped to resist persuasive but incorrect reasoning.
Robustness and Invariance of Hybrid Metaheuristics under Objective Function Transformations
Venue: arXiv cs.NE 2025
Year: 2025

This paper evaluates the robustness and structural invariance of hybrid population-based metaheuristics under various objective space transformations. A lightweight plug-and-play hybridization operator is applied to nineteen state-of-the-art algorithms-including differential evolution (DE), particle swarm optimization (PSO), and recent bio-inspired methods-without modifying their internal logic. Benchmarking on the CEC-2017 suite across four dimensions (10, 30, 50, 100) is performed under five transformation types: baseline, translation, scaling, rotation, and constant shift. Statistical comparisons based on Wilcoxon and Friedman tests, Bayesian dominance analysis, and convergence trajectory profiling consistently show that differential-based hybrids (e.g., hIMODE, hSHADE, hDMSSA) maintain high accuracy, stability, and invariance under all tested deformations. In contrast, classical algorithms-especially PSO- and HHO-based variants-exhibit significant performance degradation under non-separable or distorted landscapes. The findings confirm the superiority of adaptive, structurally resilient hybrids for real-world optimization tasks subject to domain-specific transformations.
Behind the Mask: Benchmarking Camouflaged Jailbreaks in Large Language Models
Venue: arXiv cs.CR 2025
Year: 2025

Large Language Models (LLMs) are increasingly vulnerable to a sophisticated form of adversarial prompting known as camouflaged jailbreaking. This method embeds malicious intent within seemingly benign language to evade existing safety mechanisms. Unlike overt attacks, these subtle prompts exploit contextual ambiguity and the flexible nature of language, posing significant challenges to current defense systems. This paper investigates the construction and impact of camouflaged jailbreak prompts, emphasizing their deceptive characteristics and the limitations of traditional keyword-based detection methods. We introduce a novel benchmark dataset, Camouflaged Jailbreak Prompts, containing 500 curated examples (400 harmful and 100 benign prompts) designed to rigorously stress-test LLM safety protocols. In addition, we propose a multi-faceted evaluation framework that measures harmfulness across seven dimensions: Safety Awareness, Technical Feasibility, Implementation Safeguards, Harmful Potential, Educational Value, Content Quality, and Compliance Score. Our findings reveal a stark contrast in LLM behavior: while models demonstrate high safety and content quality with benign inputs, they exhibit a significant decline in performance and safety when confronted with camouflaged jailbreak attempts. This disparity underscores a pervasive vulnerability, highlighting the urgent need for more nuanced and adaptive security strategies to ensure the responsible and robust deployment of LLMs in real-world applications.
Self-Aligned Reward: Towards Effective and Efficient Reasoners
Venue: arXiv cs.LG 2025
Year: 2025

Reinforcement learning with verifiable rewards has significantly advanced reasoning in large language models (LLMs), but such signals remain coarse, offering only binary correctness feedback. This limitation often results in inefficiencies, including overly verbose reasoning and high computational cost, while existing solutions often compromise accuracy. To address this, we introduce self-aligned reward (SAR), a self-guided signal that complements verifiable rewards to encourage both reasoning accuracy and efficiency. SAR is defined as the relative perplexity difference between an answer conditioned on the query and the standalone answer, thereby favoring responses that are concise and query-specific. Quantitative analysis reveals that SAR reliably distinguishes answer quality: concise, correct answers score higher than redundant ones, and partially correct answers score higher than entirely incorrect ones. Evaluation on 4 models across 7 benchmarks shows that integrating SAR with prevalent RL algorithms like PPO and GRPO improves accuracy by 4%, while reducing inference cost by 30%. Further analysis demonstrates that SAR achieves a Pareto-optimal trade-off between correctness and efficiency compared to reward signals based on length or self-confidence. We also show that SAR shortens responses while preserving advanced reasoning behaviors, demonstrating its ability to suppress unnecessary elaboration without losing critical reasoning. These results highlight the promise of self-aligned reward as a fine-grained complement to verifiable rewards, paving the way for more efficient and effective LLM training.
Efficient Fault Localization in a Cloud Stack Using End-to-End Application Service Topology
Venue: arXiv cs.PF 2025
Year: 2025

Cloud application services are distributed in nature and have components across the stack working together to deliver the experience to end users. The wide adoption of microservice architecture exacerbates failure management due to increased service components. To be effective, the strategies to enhance the application service resilience need to be autonomous and developed at the service's granularity, considering its end-to-end components. However, the massive amount of observability data generated by all these components across the service stack poses a significant challenge in reacting to anomalies and restoring the service quality in real time. Identifying the most informative observability data from across the cloud service stack and timely localization of root causes of anomalies thus becomes crucial to ensure service resilience. This article presents a novel approach that considers the application service topology to select the most informative metrics across the cloud stack to support efficient, explainable, and accurate root cause identifications in case of performance anomalies. The usefulness of the selected metrics is then evaluated using the state-of-the-art Root Cause Detection (RCD) algorithm for localizing the root cause of performance anomalies. As a step towards improving the accuracy and efficiency of RCD, this article then proposes the Topology-Aware-RCD (TA-RCD) that incorporates the end-to-end application service topology in RCD. The evaluation of the failure injection studies shows that the proposed approach performs at least 2X times better on average than the state-of-the-art RCD algorithm regarding Top-3 and Top-5 recall.
DreamPRM-1.5: Unlocking the Potential of Each Instance for Multimodal Process Reward Model Training
Venue: arXiv cs.LG 2025
Year: 2025

Training multimodal process reward models (PRMs) is hard due to (i) distribution shift between training set and test set and (ii) quality imbalance across training data samples. While domain-level reweighting (e.g., DreamPRM) aligns training with test-time objectives, it leaves a clear gap to an oracle upper bound (pass@N), even under a "sanity check" that uses test set data to probe headroom -- pointing to meta-level under-parameterization. We introduce DreamPRM-1.5, an instance-level reweighting framework that assigns an adaptive weight to every training example via bi-level optimization. To realize instance reweighting across scales, we develop two complementary regimes: Instance Table, which learns explicit per-sample weights and excels on small/medium data, and Instance Net, a lightweight neural network that generalizes better and scales to large corpora. A practical, stable training recipe -- time-scale matching between upper/lower updates, cold-start initialization, and bounded-range weights -- prevents divergence. Integrated with test-time scaling, DreamPRM-1.5 attains 84.6 accuracy on the MMMU validation set, 31.3 accuracy on R-Bench-V and, when paired with a leading backbone (e.g., GPT-5-mini), achieves first-place results on public multimodal reasoning leaderboards. Moreover, extensive experiments, including benchmark evaluations, baseline comparisons, and a sanity check, demonstrate that DreamPRM-1.5 closes the gap toward the oracle, achieves leading performance, and trains stably.
Mitigating Spurious Correlations Between Question and Answer via Chain-of-Thought Correctness Perception Distillation
Venue: arXiv cs.CL 2025
Year: 2025

Large language models (LLMs) excel at reasoning tasks but are expensive to deploy. Thus small language models (SLMs) are fine-tuned on CoT data generated by LLMs to copy LLMs' abilities. However, these CoT data may include noisy rationales that either fail to substantiate the answers or contribute no additional information to support answer prediction, which leads SLMs to capture spurious correlations between questions and answers and compromise the quality of reasoning. In this work, we propose Chain-of-Thought Correctness Perception Distillation (CoPeD), which aims to improve the reasoning quality of the student model from the perspectives of task setting and data utilization. Firstly, we introduce a correctness-aware task setting that encourages the student model to predict answers based on correct rationales and revise them when they are incorrect. This setting improves the faithfulness of reasoning and allows the model to learn from its mistakes. Then, we propose a Correctness-Aware Weighted loss, which dynamically adjusts the contribution of each training instance based on the combined loss of the rationale and the answer. This strategy encourages the model to focus more on samples where the rationale offers stronger support for the correct answer. Experiments have shown that CoPeD is effective on both in-distribution (IND) and out-of-distribution (OOD) benchmark reasoning datasets.
Icon$^{2}$: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation
Venue: arXiv cs.CL 2025
Year: 2025

Large Language Models (LLMs) require high quality preference datasets to align with human preferences. However, conventional methods for constructing such datasets face significant challenges: reliance on pre-collected instructions often leads to distribution mismatches with target models, while the need for sampling multiple stochastic responses introduces substantial computational overhead. In this work, we explore a paradigm shift by leveraging inherent regulation of LLMs' representation space for efficient and tailored preference dataset construction, named Icon$^{2}$. Specifically, it first extracts layer-wise direction vectors to encode sophisticated human preferences and then uses these vectors to filter self-synthesized instructions based on their inherent consistency. During decoding, bidirectional inherent control is applied to steer token representations, enabling the precise generation of response pairs with clear alignment distinctions. Experimental results demonstrate significant improvements in both alignment and efficiency. Llama3-8B and Qwen2-7B achieve an average win rate improvement of 13.89% on AlpacaEval 2.0 and 13.45% on Arena-Hard, while reducing computational costs by up to 48.1%.