OpenRaiser/Intern-Atlas huggingface dataset:
CMCTS: A Constrained Monte Carlo Tree Search Framework for Mathematical Reasoning in Large Language Model
Venue: arXiv cs.CL 2025
Year: 2025

This paper introduces the Constrained Monte Carlo Tree Search (CMCTS) framework to enhance the mathematical reasoning capabilities of Large Language Models (LLM). By incorporating a constrained action space, Process Reward Model (PRM), and partial order rules, CMCTS effectively addresses the limitations of existing MCTS methods in terms of state space diversity and action selection rationality. Specifically, during the expansion phase, CMCTS restricts action sampling to a predefined constrained action set to increase candidate state diversity. In the simulation phase, it introduces partial order rules and PRM to optimize action selection and prevent unreasonable state transitions. Experimental results show that CMCTS performs outstandingly across multiple mathematical reasoning benchmarks. Under a zero-shot setting, a 7B-parameter model achieves an average accuracy of 83.4\%, surpassing the 72B baseline model by 4.8\%. Ablation studies demonstrate that each component of the framework is crucial for performance improvement, and their combined use fully leverages their respective strengths. Overall, the CMCTS framework provides an effective approach to enhancing LLM mathematical reasoning capabilities, supported by theoretical analysis, and offers novel insights for future reasoning tasks.
Investigating Language Preference of Multilingual RAG Systems
Venue: arXiv cs.CL 2025
Year: 2025

Multilingual Retrieval-Augmented Generation (mRAG) systems enhance language models by integrating external multilingual information to produce context-aware responses. However, mRAG systems struggle with retrieving relevant information due to linguistic variations between queries and documents, generating inconsistent responses when multilingual sources conflict. In this work, we systematically investigate language preferences in both retrieval and generation of mRAG through a series of experiments. Our analysis indicates that retrievers tend to prefer high-resource and query languages, yet this preference does not consistently improve generation performance. Moreover, we observe that generators prefer the query language or Latin scripts, leading to inconsistent outputs. To overcome these issues, we propose Dual Knowledge Multilingual RAG (DKM-RAG), a simple yet effective framework that fuses translated multilingual passages with complementary model knowledge. Empirical results demonstrate that DKM-RAG mitigates language preference in generation and enhances performance across diverse linguistic settings. Code is available at https://github.com/jeonghyunpark2002/LanguagePreference.git
LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoning
Venue: arXiv cs.CL 2025
Year: 2025

Modern large language models (LLMs) employ diverse logical inference mechanisms for reasoning, making the strategic optimization of these approaches critical for advancing their capabilities. This paper systematically investigate the comparative dynamics of inductive (System 1) versus abductive/deductive (System 2) inference in LLMs. We utilize a controlled analogical reasoning environment, varying modality (textual, visual, symbolic), difficulty, and task format (MCQ / free-text). Our analysis reveals System 2 pipelines generally excel, particularly in visual/symbolic modalities and harder tasks, while System 1 is competitive for textual and easier problems. Crucially, task format significantly influences their relative advantage, with System 1 sometimes outperforming System 2 in free-text rule-execution. These core findings generalize to broader in-context learning. Furthermore, we demonstrate that advanced System 2 strategies like hypothesis selection and iterative refinement can substantially scale LLM reasoning. This study offers foundational insights and actionable guidelines for strategically deploying logical inference to enhance LLM reasoning. Resources are available at https://github.com/HKUST-KnowComp/LogiDynamics.
Stacked Intelligent Metasurface-Based Transceiver Design for Near-Field Wideband Systems
Venue: arXiv cs.IT 2025
Year: 2025

Intelligent metasurfaces may be harnessed for realizing efficient holographic multiple-input and multiple-output (MIMO) systems, at a low hardware-cost and high energy-efficiency. As part of this family, we propose a hybrid beamforming design for stacked intelligent metasurfaces (SIM) aided wideband wireless systems relying on the near-field channel model. Specifically, the holographic beamformer is designed based on configuring the phase shifts in each layer of the SIM for maximizing the sum of the baseband eigen-channel gains of all users. To optimize the SIM phase shifts, we propose a layer-by-layer iterative algorithm for optimizing the phase shifts in each layer alternately. Then, the minimum mean square error (MMSE) transmit precoding method is employed for the digital beamformer to support multi-user access. Furthermore, the mitigation of the SIM phase tuning error is also taken into account in the digital beamformer by exploiting its statistics. The power sharing ratio of each user is designed based on the iterative waterfilling power allocation algorithm. Additionally, our analytical results indicate that the spectral efficiency attained saturates in the high signal-to-noise ratio (SNR) region due to the phase tuning error resulting from the imperfect SIM hardware quality. The simulation results show that the SIM-aided holographic MIMO outperforms the state-of-the-art (SoA) single-layer holographic MIMO in terms of its achievable rate. We further demonstrate that the near-field channel model allows the SIM-based transceiver design to support multiple users, since the spatial resources represented both by the angle domain and the distance domain can be exploited.
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
Venue: arXiv cs.CL 2025
Year: 2025

Multimodal Large Language Models (MLLMs) have expanded the capabilities of traditional language models by enabling interaction through both text and images. However, ensuring the safety of these models remains a significant challenge, particularly in accurately identifying whether multimodal content is safe or unsafe-a capability we term safety awareness. In this paper, we introduce MMSafeAware, the first comprehensive multimodal safety awareness benchmark designed to evaluate MLLMs across 29 safety scenarios with 1500 carefully curated image-prompt pairs. MMSafeAware includes both unsafe and over-safety subsets to assess models abilities to correctly identify unsafe content and avoid over-sensitivity that can hinder helpfulness. Evaluating nine widely used MLLMs using MMSafeAware reveals that current models are not sufficiently safe and often overly sensitive; for example, GPT-4V misclassifies 36.1% of unsafe inputs as safe and 59.9% of benign inputs as unsafe. We further explore three methods to improve safety awareness-prompting-based approaches, visual contrastive decoding, and vision-centric reasoning fine-tuning-but find that none achieve satisfactory performance. Our findings highlight the profound challenges in developing MLLMs with robust safety awareness, underscoring the need for further research in this area. All the code and data will be publicly available to facilitate future research.
TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking
Venue: arXiv cs.CL 2025
Year: 2025

In this paper, we present TituLLMs, the first large pretrained Bangla LLMs, available in 1b and 3b parameter sizes. Due to computational constraints during both training and inference, we focused on smaller models. To train TituLLMs, we collected a pretraining dataset of approximately ~37 billion tokens. We extended the Llama-3.2 tokenizer to incorporate language- and culture-specific knowledge, which also enables faster training and inference. There was a lack of benchmarking datasets to benchmark LLMs for Bangla. To address this gap, we developed five benchmarking datasets. We benchmarked various LLMs, including TituLLMs, and demonstrated that TituLLMs outperforms its initial multilingual versions. However, this is not always the case, highlighting the complexities of language adaptation. Our work lays the groundwork for adapting existing multilingual open models to other low-resource languages. To facilitate broader adoption and further research, we have made the TituLLMs models and benchmarking datasets publicly available (https://huggingface.co/collections/hishab/titulm-llama-family-6718d31fc1b83529276f490a).
How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training
Venue: arXiv cs.LG 2025
Year: 2025

Despite exceptional capabilities in knowledge-intensive tasks, Large Language Models (LLMs) face a critical gap in understanding how they internalize new knowledge, particularly how to structurally embed acquired knowledge in their neural computations. We address this issue through the lens of knowledge circuit evolution, identifying computational subgraphs that facilitate knowledge storage and processing. Our systematic analysis of circuit evolution throughout continual pre-training reveals several key findings: (1) the acquisition of new knowledge is influenced by its relevance to pre-existing knowledge; (2) the evolution of knowledge circuits exhibits a distinct phase shift from formation to optimization; (3) the evolution of knowledge circuits follows a deep-to-shallow pattern. These insights not only advance our theoretical understanding of the mechanisms of new knowledge acquisition in LLMs, but also provide potential implications for improving continual pre-training strategies to enhance model performance. Code and data will be available at https://github.com/zjunlp/DynamicKnowledgeCircuits.
Artifact Validity in Design Science Research (DSR): A Comparative Analysis of Three Influential Frameworks
Venue: arXiv cs.OH 2025
Year: 2025

Although the methodology of Design Science Research (DSR) is playing an increasingly important role with the emergence of the "sciences of the artificial", the validity of the resulting artifacts is occasionally questioned. This paper compares three influential DSR frameworks to assess their support for artifact validity. Using five essential validity types (instrument validity, technical validity, design validity, purpose validity and generalization), the qualitative analysis reveals that while purpose validity is explicitly emphasized, instrument and design validity remain the least developed. Their implicit treatment in all frameworks poses a risk of overlooked validation, and the absence of mandatory instrument validity can lead to invalid artifacts, threatening research credibility. Beyond these findings, the paper contributes (a) a comparative overview of each framework's strengths and weaknesses and (b) a revised DSR framework incorporating all five validity types with definitions and examples. This ensures systematic artifact evaluation and improvement, reinforcing the rigor of DSR.
Deep Contrastive Learning for Feature Alignment: Insights from Housing-Household Relationship Inference
Venue: arXiv cs.LG 2025
Year: 2025

Housing and household characteristics are key determinants of social and economic well-being, yet our understanding of their interrelationships remains limited. This study addresses this knowledge gap by developing a deep contrastive learning (DCL) model to infer housing-household relationships using the American Community Survey (ACS) Public Use Microdata Sample (PUMS). More broadly, the proposed model is suitable for a class of problems where the goal is to learn joint relationships between two distinct entities without explicitly labeled ground truth data. Our proposed dual-encoder DCL approach leverages co-occurrence patterns in PUMS and introduces a bisect K-means clustering method to overcome the absence of ground truth labels. The dual-encoder DCL architecture is designed to handle the semantic differences between housing (building) and household (people) features while mitigating noise introduced by clustering. To validate the model, we generate a synthetic ground truth dataset and conduct comprehensive evaluations. The model further demonstrates its superior performance in capturing housing-household relationships in Delaware compared to state-of-the-art methods. A transferability test in North Carolina confirms its generalizability across diverse sociodemographic and geographic contexts. Finally, the post-hoc explainable AI analysis using SHAP values reveals that tenure status and mortgage information play a more significant role in housing-household matching than traditionally emphasized factors such as the number of persons and rooms.
MaskFlow: Discrete Flows For Flexible and Efficient Long Video Generation
Venue: arXiv cs.CV 2025
Year: 2025

Generating long, high-quality videos remains a challenge due to the complex interplay of spatial and temporal dynamics and hardware limitations. In this work, we introduce MaskFlow, a unified video generation framework that combines discrete representations with flow-matching to enable efficient generation of high-quality long videos. By leveraging a frame-level masking strategy during training, MaskFlow conditions on previously generated unmasked frames to generate videos with lengths ten times beyond that of the training sequences. MaskFlow does so very efficiently by enabling the use of fast Masked Generative Model (MGM)-style sampling and can be deployed in both fully autoregressive as well as full-sequence generation modes. We validate the quality of our method on the FaceForensics (FFS) and Deepmind Lab (DMLab) datasets and report Frechet Video Distance (FVD) competitive with state-of-the-art approaches. We also provide a detailed analysis on the sampling efficiency of our method and demonstrate that MaskFlow can be applied to both timestep-dependent and timestep-independent models in a training-free manner.
Leveraging Conditional Mutual Information to Improve Large Language Model Fine-Tuning For Classification
Venue: arXiv cs.CL 2025
Year: 2025

Although large language models (LLMs) have demonstrated remarkable capabilities in recent years, the potential of information theory (IT) to enhance LLM development remains underexplored. This paper introduces the information theoretic principle of Conditional Mutual Information (CMI) to LLM fine-tuning for classification tasks, exploring its promise in two main ways: minimizing CMI to improve a model's standalone performance and maximizing CMI to enhance knowledge distillation (KD) for more capable student models. To apply CMI in LLM fine-tuning, we adapt the recently proposed CMI-constrained deep learning framework, which was initially developed for image classification, with some modification. By minimizing CMI during LLM fine-tuning, we achieve superior performance gains on 6 of 8 GLUE classification tasks compared to BERT. Additionally, maximizing CMI during the KD process results in significant performance improvements in 6 of 8 GLUE classification tasks compared to DistilBERT. These findings demonstrate CMI's adaptability for optimizing both standalone LLMs and student models, showcasing its potential as a robust framework for advancing LLM fine-tuning. Our work bridges the gap between information theory and LLM development, offering new insights for building high-performing language models.
Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM's Nest
Venue: arXiv cs.CL 2025
Year: 2025

Massive high-quality data, both pre-training raw texts and post-training annotations, have been carefully prepared to incubate advanced large language models (LLMs). In contrast, for information extraction (IE), pre-training data, such as BIO-tagged sequences, are hard to scale up. We show that IE models can act as free riders on LLM resources by reframing next-token \emph{prediction} into \emph{extraction} for tokens already present in the context. Specifically, our proposed next tokens extraction (NTE) paradigm learns a versatile IE model, \emph{Cuckoo}, with 102.6M extractive data converted from LLM's pre-training and post-training data. Under the few-shot setting, Cuckoo adapts effectively to traditional and complex instruction-following IE with better performance than existing pre-trained IE models. As a free rider, Cuckoo can naturally evolve with the ongoing advancements in LLM data preparation, benefiting from improvements in LLM training pipelines without additional manual effort.
"When I lost it, they dragged me out": How Care Encounters Empower Marginalized Young Adults' Aspiration and Mental Health Care-Seeking
Venue: arXiv cs.HC 2025
Year: 2025

Mental health care-seeking among marginalized young adults has received limited attention in CSCW research. Through in-depth interviews and visual elicitation methods with 18 diverse U.S. participants, our study reveals how marginalized identities shape mental health care-seeking journeys, often characterized by low aspirations and passive care-seeking influenced by lived experiences of marginalization. However, we found the transformative function of "care encounters" - serendipitous interactions with mental health resources that occur when individuals are not actively seeking support. These encounters serve as critical turning points, catalyzing shifts in aspiration and enabling more proactive care-seeking behaviors. Our analysis identifies both the infrastructural conditions that enable transformative care encounters and the aspiration breakdowns that impede care-seeking processes. This work makes conceptual contributions by supplementing traditional motivation-based care-seeking models with a reconceptualization of "care encounters" that accounts for the infrastructural and serendipitous nature of mental health access. We advance understanding of how marginalized identity uniquely influences care-seeking behaviors while providing actionable design implications for embedding technology-mediated "care encounters" into socio-technical interventions that can better support mental health care access for vulnerable populations.
Grassroots Platforms with Atomic Transactions: Social Networks, Cryptocurrencies, and Democratic Federations
Venue: arXiv cs.DC 2025
Year: 2025

Grassroots platforms aim to offer an egalitarian alternative to global platforms. Whereas global platforms can have only a single instance, grassroots platforms can have multiple instances that emerge and operate independently of each other and of any global resource except the network, and can interoperate and coalesce into ever-larger instances once interconnected. Key grassroots platforms include grassroots social networks, grassroots cryptocurrencies, and grassroots democratic federations. Previously, grassroots platforms were defined formally and proven grassroots using unary distributed transition systems, in which each transition is carried out by a single agent. However, grassroots platforms cater for a more abstract specification using transactions carried out atomically by multiple agents, something that cannot be expressed by unary transition systems. As a result, their original specifications and proofs were unnecessarily cumbersome and opaque. We enhance the notion of a distributed transition system to include atomic transactions and revisit the notion of grassroots platforms within this new foundation; present crisp specifications of key grassroots platforms using atomic transactions: befriending and defriending for grassroots social networks, coin swaps for grassroots cryptocurrencies, and communities forming, joining, and leaving a federation for grassroots democratic federations; prove a general theorem that a platform specified by atomic transactions that are so-called interactive is grassroots; show that the atomic transactions used to specify all three platforms are interactive; and conclude that the platforms thus specified are indeed grassroots. We thus provide a crisp mathematical foundation for grassroots platforms and a solid and clear starting point from which their implementation can commence.
Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation
Venue: arXiv cs.CL 2025
Year: 2025

Large language models (LLMs) often suffer from hallucination, generating factually incorrect or ungrounded content, which limits their reliability in high-stakes applications. A key factor contributing to hallucination is the use of hard labels during training, which enforce deterministic supervision, encourage overconfidence, and disregard the uncertainty inherent in natural language. To address this, we propose mitigating hallucination through knowledge distillation (KD), where a teacher model provides smoothed soft labels to a student model, reducing overconfidence and improving factual grounding. We apply KD during supervised finetuning on instructional data, evaluating its effectiveness across LLMs from different families. Experimental results on summarization benchmarks demonstrate that KD reduces hallucination compared to standard finetuning while preserving performance on general NLP tasks. These findings highlight KD as a promising approach for mitigating hallucination in LLMs and improving model reliability.
WRT-SAM: Foundation Model-Driven Segmentation for Generalized Weld Radiographic Testing
Venue: arXiv cs.CV 2025
Year: 2025

Radiographic testing is a fundamental non-destructive evaluation technique for identifying weld defects and assessing quality in industrial applications due to its high-resolution imaging capabilities. Over the past decade, deep learning techniques have significantly advanced weld defect identification in radiographic images. However, conventional approaches, which rely on training small-scale, task-specific models on single-scenario datasets, exhibit poor cross-scenario generalization. Recently, the Segment Anything Model (SAM), a pre-trained visual foundation model trained on large-scale datasets, has demonstrated exceptional zero-shot generalization capabilities. Fine-tuning SAM with limited domain-specific data has yielded promising results in fields such as medical image segmentation and anomaly detection. To the best of our knowledge, this work is the first to introduce SAM-based segmentation for general weld radiographic testing images. We propose WRT-SAM, a novel weld radiographic defect segmentation model that leverages SAM through an adapter-based integration with a specialized prompt generator architecture. To improve adaptability to grayscale weld radiographic images, we introduce a frequency prompt generator module, which enhances the model's sensitivity to frequency-domain information. Furthermore, to address the multi-scale nature of weld defects, we incorporate a multi-scale prompt generator module, enabling the model to effectively extract and encode defect information across varying scales. Extensive experimental evaluations demonstrate that WRT-SAM achieves a recall of 78.87%, a precision of 84.04%, and an AUC of 0.9746, setting a new state-of-the-art (SOTA) benchmark. Moreover, the model exhibits superior zero-shot generalization performance, highlighting its potential for practical deployment in diverse radiographic testing scenarios.