OpenRaiser/Intern-Atlas huggingface dataset:
MORepair: Teaching LLMs to Repair Code via Multi-Objective Fine-tuning
Venue: arXiv cs.SE 2025
Year: 2025

Within the realm of software engineering, specialized tasks on code, such as program repair, present unique challenges, necessitating fine-tuning Large language models~(LLMs) to unlock state-of-the-art performance. Fine-tuning approaches proposed in the literature for LLMs on program repair tasks generally overlook the need to reason about the logic behind code changes, beyond syntactic patterns in the data. High-performing fine-tuning experiments also usually come at very high computational costs. With MORepair, we propose a novel perspective on the learning focus of LLM fine-tuning for program repair: we not only adapt the LLM parameters to the syntactic nuances of the task of code transformation (objective 1), but we also specifically fine-tune the LLM with respect to the logical reason behind the code change in the training data (objective 2). Such a multi-objective fine-tuning will instruct LLMs to generate high-quality patches. We apply MORepair to fine-tune four open-source LLMs with different sizes and architectures. Experimental results on function-level and repository-level repair benchmarks show that the implemented fine-tuning effectively boosts LLM repair performance by 11.4% to 56.0%. We further show that our fine-tuning strategy yields superior performance compared to the state-of-the-art approaches, including standard fine-tuning, Fine-tune-CoT, and RepairLLaMA.
LiMe: a Latin Corpus of Late Medieval Criminal Sentences
Venue: arXiv cs.CL 2025
Year: 2025

The Latin language has received attention from the computational linguistics research community, which has built, over the years, several valuable resources, ranging from detailed annotated corpora to sophisticated tools for linguistic analysis. With the recent advent of large language models, researchers have also started developing models capable of generating vector representations of Latin texts. The performances of such models remain behind the ones for modern languages, given the disparity in available data. In this paper, we present the LiMe dataset, a corpus of 325 documents extracted from a series of medieval manuscripts called Libri sententiarum potestatis Mediolani, and thoroughly annotated by experts, in order to be employed for masked language model, as well as supervised natural language processing tasks.
Unlocking Multi-View Insights in Knowledge-Dense Retrieval-Augmented Generation
Venue: arXiv cs.CL 2025
Year: 2025

While Retrieval-Augmented Generation (RAG) plays a crucial role in the application of Large Language Models (LLMs), existing retrieval methods in knowledge-dense domains like law and medicine still suffer from a lack of multi-perspective views, which are essential for improving interpretability and reliability. Previous research on multi-view retrieval often focused solely on different semantic forms of queries, neglecting the expression of specific domain knowledge perspectives. This paper introduces a novel multi-view RAG framework, MVRAG, tailored for knowledge-dense domains that utilizes intention-aware query rewriting from multiple domain viewpoints to enhance retrieval precision, thereby improving the effectiveness of the final inference. Experiments conducted on legal and medical case retrieval demonstrate significant improvements in recall and precision rates with our framework. Our multi-perspective retrieval approach unleashes the potential of multi-view information enhancing RAG tasks, accelerating the further application of LLMs in knowledge-intensive fields.
MM-PhyRLHF: Reinforcement Learning Framework for Multimodal Physics Question-Answering
Venue: arXiv cs.AI 2025
Year: 2025

Recent advancements in LLMs have shown their significant potential in tasks like text summarization and generation. Yet, they often encounter difficulty while solving complex physics problems that require arithmetic calculation and a good understanding of concepts. Moreover, many physics problems include images that contain important details required to understand the problem's context. We propose an LMM-based chatbot to answer multimodal physics MCQs. For domain adaptation, we utilize the MM-PhyQA dataset comprising Indian high school-level multimodal physics problems. To improve the LMM's performance, we experiment with two techniques, RLHF (Reinforcement Learning from Human Feedback) and Image Captioning. In image captioning, we add a detailed explanation of the diagram in each image, minimizing hallucinations and image processing errors. We further explore the integration of Reinforcement Learning from Human Feedback (RLHF) methodology inspired by the ranking approach in RLHF to enhance the human-like problem-solving abilities of the models. The RLHF approach incorporates human feedback into the learning process of LLMs, improving the model's problem-solving skills, truthfulness, and reasoning capabilities, minimizing the hallucinations in the answers, and improving the quality instead of using vanilla-supervised fine-tuned models. We employ the LLaVA open-source model to answer multimodal physics MCQs and compare the performance with and without using RLHF.
Strengthening Infrastructure Resilience to Hurricanes by Modeling Transportation and Electric Power Network Interdependencies
Venue: arXiv cs.CE 2025
Year: 2025

This study presents an agent-based model (ABM) developed to simulate the resilience of a community to hurricane-induced infrastructure disruptions, focusing on the interdependencies between electric power and transportation networks. In this ABM approach, agents represent the components of a system, where interactions within a system shape intra-dependency of a system and interactions among systems shape interdependencies. To study household resilience subject to a hurricane, a library of agents has been created including electric power network, transportation network, wind/flooding hazards, and household agents. The ABM is applied over the household and infrastructure data from a community (Zip code 33147) in Miami-Dade County, Florida. Interdependencies between the two networks are modeled in two ways, (i) representing the role of transportation in fuel delivery to power plants and restoration teams' access, (ii) impact of power outage on transportation network components. Restoring traffic signals quickly is crucial as their outage can slow down traffic and increase the chance of crashes. We simulate three restoration strategies: component based, distance based, and traffic lights based restoration. The model is validated against Hurricane Irma data, showing consistent behavior with varying hazard intensities. Scenario analyses explore the impact of restoration strategies, road accessibility, and wind speed intensities on power restoration. Results demonstrate that a traffic lights based restoration strategy efficiently prioritizes signal recovery without delaying household power restoration time. Restoration of power services will be faster if restoration teams do not need to wait due to inaccessible roads and fuel transportation to power plants is not delayed.
A Generative Approach to Credit Prediction with Learnable Prompts for Multi-scale Temporal Representation Learning
Venue: arXiv cs.CE 2025
Year: 2025

Recent industrial credit scoring models remain heavily reliant on manually tuned statistical learning methods. Despite their potential, deep learning architectures have struggled to consistently outperform traditional statistical models in industrial credit scoring, largely due to the complexity of heterogeneous financial data and the challenge of modeling evolving creditworthiness. To bridge this gap, we introduce FinLangNet, a novel framework that reformulates credit scoring as a multi-scale sequential learning problem. FinLangNet processes heterogeneous financial data through a dual-module architecture that combines tabular feature extraction with temporal sequence modeling, generating probability distributions of users' future financial behaviors across multiple time horizons. A key innovation is our dual-prompt mechanism within the sequential module, which introduces learnable prompts operating at both feature-level granularity for capturing fine-grained temporal patterns and user-level granularity for aggregating holistic risk profiles. In extensive evaluations, FinLangNet significantly outperforms a production XGBoost system, achieving a 7.2% improvement in the KS metric and a 9.9% relative reduction in bad debt rate. Its effectiveness as a general-purpose sequential learning framework is further validated through state-of-the-art performance on the public UEA time series classification benchmark. The system has been successfully deployed on DiDi's international finance platform, serving leading financial credit companies in Latin America.
Reduction systems and degree bounds for integration
Venue: arXiv cs.SC 2025
Year: 2025

In symbolic integration, the Risch--Norman algorithm aims to find closed forms of elementary integrals over differential fields by an ansatz for the integral, which usually is based on heuristic degree bounds. Norman presented an approach that avoids degree bounds and only relies on the completion of reduction systems. We give a formalization of his approach and we develop a refined completion process, which terminates in more instances. In some situations when the completion process does not terminate, one can detect patterns allowing to still describe infinite reduction systems that are complete. We present such infinite systems for the fields generated by Airy functions and complete elliptic integrals, respectively. Moreover, we show how complete reduction systems can be used to find rigorous degree bounds. In particular, we give a general formula for weighted degree bounds and we apply it to find tight bounds in the above examples.
An FPTAS for Shortest-Longest Path Problem
Venue: arXiv cs.NI 2025
Year: 2025

Motivated by multi-domain service function chain (SFC) orchestration, we define the shortest-longest path (SLP) problem, prove its hardness, and design an efficient fully polynomial time approximation scheme (FPTAS) using the dynamic programming (DP) and scaling and rounding (SR) techniques to compute an approximation solution with provable performance guarantee. The SLP problem and its solution algorithm have theoretical significance in multicriteria optimization and also have application potential in QoS routing and multi-domain network resource allocation scenarios.
Machine Vision-Based Assessment of Fall Color Changes and its Relationship with Leaf Nitrogen Concentration
Venue: arXiv cs.CV 2025
Year: 2025

Apple(\textit{Malus domestica} Borkh.) trees are deciduous, shedding leaves each year. This process is preceded by a gradual change in leaf color from green to yellow as chlorophyll is degraded prior to abscission. The initiation and rate of this color change are affected by many factors including leaf nitrogen (N) concentration. We predict that leaf color during this transition may be indicative of the nitrogen status of apple trees. This study assesses a machine vision-based system for quantifying the change in leaf color and its correlation with leaf nitrogen content. An image dataset was collected in color and 3D over five weeks in the fall of 2021 and 2023 at a commercial orchard using a ground vehicle-based stereovision sensor. Trees in the foreground were segmented from the point cloud using color and depth thresholding methods. Then, to estimate the proportion of yellow leaves per canopy, the color information of the segmented canopy area was quantified using a custom-defined metric, \textit{yellowness index} (a normalized ratio of yellow to green foliage in the tree) that varied from -1 to +1 (-1 being completely green and +1 being completely yellow). Both K-means-based methods and gradient boosting methods were used to estimate the \textit{yellowness index}. The gradient boosting based method proposed in this study was better than the K-means-based method (both in terms of computational time and accuracy), achieving an $R^2$ of 0.72 in estimating the \textit{yellowness index}. The metric was able to capture the gradual color transition from green to yellow over the study duration. Trees with lower leaf nitrogen showed the color transition to yellow earlier than the trees with higher nitrogen. Keywords: Fruit Tree Nitrogen Management, Machine Vision, Point Cloud Segmentation, Precision Nitrogen Management
Vul-LMGNNs: Fusing language models and online-distilled graph neural networks for code vulnerability detection
Venue: arXiv cs.CR 2025
Year: 2025

Code Language Models (codeLMs) and Graph Neural Networks (GNNs) are widely used in code vulnerability detection. However, GNNs often rely on aggregating information from adjacent nodes, limiting structural information propagation across layers. While codeLMs can supplement GNNs with semantic information, existing integration methods underexplore their collaborative potential. To address these challenges, we propose Vul-LMGNNs, integrating pre-trained codeLMs with GNNs to enable cross-layer propagation of semantic and structural information. Vul-LMGNNs leverage Code Property Graphs (CPGs) to incorporate syntax, control flow, and data dependencies, using gated GNNs for structural extraction. An online knowledge distillation (KD) mechanism allows a student GNN to capture structural information from a trained counterpart via alternating training. Additionally, an "implicit-explicit" joint training framework leverages codeLMs to initialize embeddings and propagate code semantics. In the explicit phase, it performs late fusion via linear interpolation. Evaluations on real-world vulnerability datasets show Vul-LMGNNs outperform 17 state-of-the-art approaches. Source code is available at: https://github.com/Vul-LMGNN/vul-LMGNN.
SST: Multi-Scale Hybrid Mamba-Transformer Experts for Time Series Forecasting
Venue: arXiv cs.LG 2025
Year: 2025

Time series forecasting has made significant advances, including with Transformer-based models. The attention mechanism in Transformer effectively captures temporal dependencies by attending to all past inputs simultaneously. However, its quadratic complexity with respect to sequence length limits the scalability for long-range modeling. Recent state space models (SSMs) such as Mamba offer a promising alternative by achieving linear complexity without attention. Yet, Mamba compresses historical information into a fixed-size latent state, potentially causing information loss and limiting representational effectiveness. This raises a key research question: Can we design a hybrid Mamba-Transformer architecture that is both effective and efficient for time series forecasting? To address it, we adapt a hybrid Mamba-Transformer architecture Mambaformer, originally proposed for language modeling, to the time series domain. Preliminary experiments reveal that naively stacking Mamba and Transformer layers in Mambaformer is suboptimal for time series forecasting, due to an information interference problem. To mitigate this issue, we introduce a new time series decomposition strategy that separates time series into long-range patterns and short-range variations. Then we show that Mamba excels at capturing long-term structures, while Transformer is more effective at modeling short-term dynamics. Building on this insight, we propose State Space Transformer (SST), a multi-scale hybrid model with expert modules: a Mamba expert for long-range patterns and a Transformer expert for short-term variations. SST also employs a multi-scale patching mechanism to adaptively adjust time series resolution: low resolution for long-term patterns and high resolution for short-term variations. Experiments show that SST obtains SOTA performance with linear scalability. The code is at https://github.com/XiongxiaoXu/SST.
GLDPC-PC Codes: Channel Coding Towards 6G Communications
Venue: arXiv cs.IT 2025
Year: 2025

The sixth generation (6G) wireless communication system will improve the key technical indicators by one to two orders of magnitude, and come with some new features. As a crucial technique to enhance the reliability and efficiency of data transmission, the next-generation channel coding is thus confronted with new challenges in terms of complexity, latency, performance. This article supplies an overview of the potential channel codes for 6G communications. In addition, we explore to develop next-generation channel codes based on lowdensity parity-check (LDPC) and polar frameworks, introducing a concept named generalized LDPC with polar-like component(GLDPC-PC) codes. The codes have exhibited promising error correction performance and manageable complexity, which can be further optimized by specific code design. The opportunities and challenges of GLDPC-PC codes are also discussed.
Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems
Venue: arXiv cs.CL 2025
Year: 2025

Chain-of-Thought (CoT) prompting has enhanced the performance of Large Language Models (LLMs) across various reasoning tasks. However, CoT still falls short in dealing with complex math word problems, as it usually suffers from three pitfalls: semantic misunderstanding errors, calculation errors, and step-missing errors. Prior studies involve addressing the calculation errors and step-missing errors, but neglect the semantic misunderstanding errors, which is the major factor limiting the reasoning performance of LLMs. To this end, we propose a simple-yet-effective method, namely Deeply Understanding the Problems (DUP), to improve the LLMs' math problem-solving ability by addressing semantic misunderstanding errors. The core of our method is to encourage the LLMs to deeply understand the problems and extract the key problem-solving information used for better reasoning. Extensive experiments on 10 diverse reasoning benchmarks show that our DUP method consistently outperforms the other counterparts by a large margin. More encouragingly, DUP achieves a new SOTA result on the GSM8K benchmark, with an accuracy of 97.1% under the zero-shot setting.
Measuring Diversity of Game Scenarios
Venue: arXiv cs.AI 2025
Year: 2025

This survey comprehensively reviews the multi-dimensionality of game scenario diversity, spotlighting the innovative use of procedural content generation and other fields as cornerstones for enriching player experiences through diverse game scenarios. By traversing a wide array of disciplines, from affective modeling and multi-agent systems to psychological studies, our research underscores the importance of diverse game scenarios in gameplay and education. Through a taxonomy of diversity metrics and evaluation methods, we aim to bridge the current gaps in literature and practice, offering insights into effective strategies for measuring and integrating diversity in game scenarios. Our analysis highlights the necessity for a unified taxonomy to aid developers and researchers in crafting more engaging and varied game worlds. This survey not only charts a path for future research in diverse game scenarios but also serves as a handbook for industry practitioners seeking to leverage diversity as a key component of game design and development.
Lovász Theorems for Modal Languages
Venue: arXiv cs.LO 2025
Year: 2025

A famous result due to Lovász states that two finite relational structures $M$ and $N$ are isomorphic if, and only if, for all finite relational structures $T$, the number of homomorphisms from $T$ to $M$ is equal to the number of homomorphisms from $T$ to $N$. Since first-order logic (FOL) can describe finite structures up to isomorphism, this can be interpreted as a characterization of FOL-equivalence via homomorphism-count indistinguishability with respect to the class of finite structures. We identify classes of labeled transition systems (LTSs) such that homomorphism-count indistinguishability with respect to these classes, where "counting" is done within an appropriate semiring structure, captures equivalence with respect to positive-existential modal logic, graded modal logic, and hybrid logic, as well as the extensions of these logics with either backward or global modalities. Our positive results apply not only to finite structures, but also to certain well-behaved infinite structures. We also show that equivalence with respect to positive modal logic and equivalence with respect to the basic modal language are not captured by homomorphism-count indistinguishability with respect to any class of LTSs, regardless of which semiring is used for counting.
MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysis
Venue: arXiv cs.CV 2025
Year: 2025

The Vision Transformer (ViT) has demonstrated remarkable performance in Self-Supervised Learning (SSL) for 3D medical image analysis. Masked AutoEncoder (MAE) for feature pre-training can further unleash the potential of ViT on various medical vision tasks. However, due to large spatial sizes with much higher dimensions of 3D medical images, the lack of hierarchical design for MAE may hinder the performance of downstream tasks. In this paper, we propose a novel \textit{Mask in Mask (MiM)} pre-training framework for 3D medical images, which aims to advance MAE by learning discriminative representation from hierarchical visual tokens across varying scales. We introduce multiple levels of granularity for masked inputs from the volume, which are then reconstructed simultaneously ranging at both fine and coarse levels. Additionally, a cross-level alignment mechanism is applied to adjacent level volumes to enforce anatomical similarity hierarchically. Furthermore, we adopt a hybrid backbone to enhance the hierarchical representation learning efficiently during the pre-training. MiM was pre-trained on a large scale of available 3D volumetric images, \textit{i.e.,} Computed Tomography (CT) images containing various body parts. Extensive experiments on thirteen public datasets demonstrate the superiority of MiM over other SSL methods in organ/lesion/tumor segmentation and disease classification. We further scale up the MiM to large pre-training datasets with more than 10k volumes, showing that large-scale pre-training can further enhance the performance of downstream tasks. The improvement also concluded that the research community should pay more attention to the scale of the pre-training dataset towards the healthcare foundation model for 3D medical images.
Effective and General Distance Computation for Approximate Nearest Neighbor Search
Venue: arXiv cs.DB 2025
Year: 2025

Approximate K Nearest Neighbor (AKNN) search in high-dimensional spaces is a critical yet challenging problem. In AKNN search, distance computation is the core task that dominates the runtime. Existing approaches typically use approximate distances to improve computational efficiency, often at the cost of reduced search accuracy. To address this issue, the state-of-the-art method, ADSampling, employs random projections to estimate approximate distances and introduces an additional distance correction process to mitigate accuracy loss. However, ADSampling has limitations in both effectiveness and generality, primarily due to its reliance on random projections for distance approximation and correction. To address the effectiveness limitations of ADSampling, we leverage data distribution to improve distance computation via orthogonal projection. Furthermore, to overcome the generality limitations of ADSampling, we adopt a data-driven approach to distance correction, decoupling the correction process from the distance approximation process. Extensive experiments demonstrate the superiority and effectiveness of our method. In particular, compared to ADSampling, our method achieves a speedup of 1.6 to 2.1 times on real-world datasets while providing higher accuracy.