OpenRaiser/Intern-Atlas huggingface dataset:
LLM4GNAS: A Large Language Model Based Toolkit for Graph Neural Architecture Search
Venue: arXiv cs.LG 2025
Year: 2025

Graph Neural Architecture Search (GNAS) facilitates the automatic design of Graph Neural Networks (GNNs) tailored to specific downstream graph learning tasks. However, existing GNAS approaches often require manual adaptation to new graph search spaces, necessitating substantial code optimization and domain-specific knowledge. To address this challenge, we present LLM4GNAS, a toolkit for GNAS that leverages the generative capabilities of Large Language Models (LLMs). LLM4GNAS includes an algorithm library for graph neural architecture search algorithms based on LLMs, enabling the adaptation of GNAS methods to new search spaces through the modification of LLM prompts. This approach reduces the need for manual intervention in algorithm adaptation and code modification. The LLM4GNAS toolkit is extensible and robust, incorporating LLM-enhanced graph feature engineering, LLM-enhanced graph neural architecture search, and LLM-enhanced hyperparameter optimization. Experimental results indicate that LLM4GNAS outperforms existing GNAS methods on tasks involving both homogeneous and heterogeneous graphs.
YNote: A Novel Music Notation for Fine-Tuning LLMs in Music Generation
Venue: arXiv cs.SD 2025
Year: 2025

The field of music generation using Large Language Models (LLMs) is evolving rapidly, yet existing music notation systems, such as MIDI, ABC Notation, and MusicXML, remain too complex for effective fine-tuning of LLMs. These formats are difficult for both machines and humans to interpret due to their variability and intricate structure. To address these challenges, we introduce YNote, a simplified music notation system that uses only four characters to represent a note and its pitch. YNote's fixed format ensures consistency, making it easy to read and more suitable for fine-tuning LLMs. In our experiments, we fine-tuned GPT-2 (124M) on a YNote-encoded dataset and achieved BLEU and ROUGE scores of 0.883 and 0.766, respectively. With just two notes as prompts, the model was able to generate coherent and stylistically relevant music. We believe YNote offers a practical alternative to existing music notations for machine learning applications and has the potential to significantly enhance the quality of music generation using LLMs.
Diverse Transformer Decoding for Offline Reinforcement Learning Using Financial Algorithmic Approaches
Venue: arXiv cs.AI 2025
Year: 2025

Offline Reinforcement Learning (RL) algorithms learn a policy using a fixed training dataset, which is then deployed online to interact with the environment and make decisions. Transformers, a standard choice for modeling time-series data, are gaining popularity in offline RL. In this context, Beam Search (BS), an approximate inference algorithm, is the go-to decoding method. Offline RL eliminates the need for costly or risky online data collection. However, the restricted dataset induces uncertainty as the agent may encounter unfamiliar sequences of states and actions during execution that were not covered in the training data. In this context, BS lacks two important properties essential for offline RL: It does not account for the aforementioned uncertainty, and its greedy left-right search approach often results in sequences with minimal variations, failing to explore potentially better alternatives. To address these limitations, we propose Portfolio Beam Search (PBS), a simple-yet-effective alternative to BS that balances exploration and exploitation within a Transformer model during decoding. We draw inspiration from financial economics and apply these principles to develop an uncertainty-aware diversification mechanism, which we integrate into a sequential decoding algorithm at inference time. We empirically demonstrate the effectiveness of PBS on the D4RL locomotion benchmark, where it achieves higher returns and significantly reduces outcome variability.
VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gap
Venue: arXiv cs.CR 2025
Year: 2025

The emergence of vision language models (VLMs) comes with increased safety concerns, as the incorporation of multiple modalities heightens vulnerability to attacks. Although VLMs can be built upon LLMs that have textual safety alignment, it is easily undermined when the vision modality is integrated. We attribute this safety challenge to the modality gap, a separation of image and text in the shared representation space, which blurs the distinction between harmful and harmless queries that is evident in LLMs but weakened in VLMs. To avoid safety decay and fulfill the safety alignment gap, we propose VLM-Guard, an inference-time intervention strategy that leverages the LLM component of a VLM as supervision for the safety alignment of the VLM. VLM-Guard projects the representations of VLM into the subspace that is orthogonal to the safety steering direction that is extracted from the safety-aligned LLM. Experimental results on three malicious instruction settings show the effectiveness of VLM-Guard in safeguarding VLM and fulfilling the safety alignment gap between VLM and its LLM component.
Fast Proxies for LLM Robustness Evaluation
Venue: arXiv cs.CR 2025
Year: 2025

Evaluating the robustness of LLMs to adversarial attacks is crucial for safe deployment, yet current red-teaming methods are often prohibitively expensive. We compare the ability of fast proxy metrics to predict the real-world robustness of an LLM against a simulated attacker ensemble. This allows us to estimate a model's robustness to computationally expensive attacks without requiring runs of the attacks themselves. Specifically, we consider gradient-descent-based embedding-space attacks, prefilling attacks, and direct prompting. Even though direct prompting in particular does not achieve high ASR, we find that it and embedding-space attacks can predict attack success rates well, achieving $r_p=0.87$ (linear) and $r_s=0.94$ (Spearman rank) correlations with the full attack ensemble while reducing computational cost by three orders of magnitude.
Scientific literature cited in patents: A Technology Transfer indicator in Portuguese universities
Venue: arXiv cs.DL 2025
Year: 2025

The study aims to identify the process of transfer from science to technology that occurs in the main Portuguese public universities. The methodology was based on the analysis of the scientific literature cited in patents. Data was obtained from the Lens patent database. 10,514 scientific articles cited in patents were retrieved. A descriptive analysis of the data was performed. Science maps were created to visualize the main research trends. The results showed a valuable impact of academic research in certain scientific disciplines, such as Chemistry, Biology, Materials Sciences and Medicine. The main research fronts were cancer, nanoparticles, biomaterials, tissue engineering or molecular biology. In conclusion, the research produced by Portuguese universities has generated relevant knowledge for patented inventions and the science-technology flow within specific areas.
Hallucinations and Truth: A Comprehensive Accuracy Evaluation of RAG, LoRA and DoRA
Venue: arXiv cs.CL 2025
Year: 2025

Recent advancements in Generative AI have significantly improved the efficiency and adaptability of natural language processing (NLP) systems, particularly through Retrieval-Augmented Generation (RAG), Low-Rank Adaptation (LoRA), and Weight-Decomposed Low-Rank Adaptation (DoRA). RAG integrates external knowledge to enhance factual consistency in generative outputs, while LoRA enables parameter-efficient fine-tuning of large language models (LLMs). DoRA further refines this process by optimizing fine-tuning through adaptive parameter ranking and domain-aware weight adjustments, improving learning efficiency while maintaining inference performance. This paper presents a large-scale empirical evaluation of RAG, LoRA, and DoRA, with model fine-tuning and generation performance assessed on 20,000 FAQ-based queries, while the knowledge base spans 400,000 entries. The study analyzes key performance metrics such as accuracy, relevance, and inference latency. Experimental results demonstrate that DoRA achieves the highest accuracy (90.1%), relevance score (0.88), and lowest latency (110 ms per query), outperforming both LoRA and RAG in real-world, domain-specific generative AI applications. Furthermore, this study examines the trade-offs between fine-tuning efficiency, computational cost, and real-time adaptability across different models. Findings highlight RAG's effectiveness in knowledge grounding, LoRA's cost-efficient domain adaptation, and DoRA's ability to balance fine-tuning efficiency with model precision. These insights provide practical guidance for deploying AI-driven generative systems in accuracy-critical domains such as healthcare, finance, and legal services, ensuring scalability, reliability, and optimal performance in dynamic environments.
A new lower bound for multi-color discrepancy with applications to fair division
Venue: arXiv cs.GT 2025
Year: 2025

A classical problem in combinatorics seeks colorings of low discrepancy. More concretely, the goal is to color the elements of a set system so that the number of appearances of any color among the elements in each set is as balanced as possible. We present a new lower bound for multi-color discrepancy, showing that there is a set system with $n$ subsets over a set of elements in which any $k$-coloring of the elements has discrepancy at least $Ω\left(\sqrt{\frac{n}{\ln{k}}}\right)$. This result improves the previously best-known lower bound of $Ω\left(\sqrt{\frac{n}{k}}\right)$ of Doerr and Srivastav [2003] and may have several applications. Here, we explore its implications on the feasibility of fair division concepts for instances with $n$ agents having valuations for a set of indivisible items. The first such concept is known as consensus $1/k$-division up to $d$ items (\cd$d$) and aims to allocate the items into $k$ bundles so that no matter which bundle each agent is assigned to, the allocation is envy-free up to $d$ items. The above lower bound implies that \cd$d$ can be infeasible for $d\in Ω\left(\sqrt{\frac{n}{\ln{k}}}\right)$. We furthermore extend our proof technique to show that there exist instances of the problem of allocating indivisible items to $k$ groups of $n$ agents in total so that envy-freeness and proportionality up to $d$ items are infeasible for $d\in Ω\left(\sqrt{\frac{n}{k\ln{k}}}\right)$ and $d\in Ω\left(\sqrt{\frac{n}{k^3\ln{k}}}\right)$, respectively. The lower bounds for fair division improve the currently best-known ones by Manurangsi and Suksompong [2022].
Customizable Contraction Hierarchies -- A Survey
Venue: arXiv cs.DS 2025
Year: 2025

This work establishes the technical fundamentals of a well-tuned Customizable Contraction Hierarchies (CCH) implementation that is simple and elegant. We give a detailed overview of the state of the art of CCH, review recent advances on CCH and show how to combine them. Additionally, we propose further refinements that improve the performance of CCH. An extensive evaluation confirms that a CCH framework is not only comprehensive in supported features but also competitive in performance to both Contraction Hierarchies (CH) and Customizable Route Planning (CRP).
Named entity recognition for Serbian legal documents: Design, methodology and dataset development
Venue: arXiv cs.CL 2025
Year: 2025

Recent advancements in the field of natural language processing (NLP) and especially large language models (LLMs) and their numerous applications have brought research attention to design of different document processing tools and enhancements in the process of document archiving, search and retrieval. Domain of official, legal documents is especially interesting due to vast amount of data generated on the daily basis, as well as the significant community of interested practitioners (lawyers, law offices, administrative workers, state institutions and citizens). Providing efficient ways for automation of everyday work involving legal documents is therefore expected to have significant impact in different fields. In this work we present one LLM based solution for Named Entity Recognition (NER) in the case of legal documents written in Serbian language. It leverages on the pre-trained bidirectional encoder representations from transformers (BERT), which had been carefully adapted to the specific task of identifying and classifying specific data points from textual content. Besides novel dataset development for Serbian language (involving public court rulings), presented system design and applied methodology, the paper also discusses achieved performance metrics and their implications for objective assessment of the proposed solution. Performed cross-validation tests on the created manually labeled dataset with mean $F_1$ score of 0.96 and additional results on the examples of intentionally modified text inputs confirm applicability of the proposed system design and robustness of the developed NER solution.
Post-training an LLM for RAG? Train on Self-Generated Demonstrations
Venue: arXiv cs.CL 2025
Year: 2025

Large language models (LLMs) often struggle with knowledge intensive NLP tasks, such as answering "Who won the latest World Cup?" because the knowledge they learn during training may be insufficient or outdated. Conditioning generation on retrieved documents -- a technique known as retrieval augmented generation (RAG) -- mitigates these shortcomings by allowing the model to leverage in-context information. Practitioners can improve LLM RAG performance by fine-tuning on retrieval-augmented instructions, but must beware that this can cause undesirable model behaviors like hallucinations. We attribute this degradation to the fact that the training data is likely to be out-of-distribution for the model and may suffer from quality issues, such as misalignment between retrievals and target responses (since retrievals are frequently added post-hoc). We propose a recipe for training RAG-enabled LLMs using self-generated demonstrations, thereby avoiding training on out-of-distribution text and integrating retrievals into the LLM responses. We evaluate our method on knowledge intensive question answering (QA) tasks and show that our method teaches LLMs to properly handle in-context retrievals and abstain from questions it will likely get wrong. Compared to conventional RA-IT methods, our method prevents model degradation in non-RAG settings while exhibiting superior QA performance.
Non-Negotiated Implicit ETSI VAM Clustering
Venue: arXiv cs.NI 2025
Year: 2025

Including Vulnerable Road User (VRU) in Cooperative Intelligent Transport Systems (C-ITS) framework aims to increase road safety. However, this approach implies a massive increase of network nodes and thus is vulnerable to medium capacity issues, e.g., contention, congestion, resource scheduling. Implementing cluster schemes -- to reduce the number of nodes but represent the same number of VRUs -- is a direct way to address the issue. One of them is suggested by European Telecommunications Standards Institute (ETSI) and consists of nodes (connected pedestrians and cyclists) sending vicarious messages to enable a leader node to cover for a cluster of VRUs. However, the proposed scheme includes negotiation to establish a cluster, and in-cluster communication to maintain it, requiring extra messages of variable sizes and thus does not fully resolve the original medium capacity issues. Furthermore, these exchanges assume network reliability (i.e. a lossless channel and low latency to meet time constraints). We propose a method for VRU Awareness Message (VAM) clustering where 1) all cluster operations are performed without negotiation, 2) cluster leaders do not require sending additional messages or meet deadlines, and 3) assumes a lossy communication channel and offers a mechanism for cluster resilience. Our results show the feasibility of the concept by halving message generations compared to individual messages while keeping the awareness levels (i.e., that VRUs are accounted for).
ProMRVL-CAD: Proactive Dialogue System with Multi-Round Vision-Language Interactions for Computer-Aided Diagnosis
Venue: arXiv cs.AI 2025
Year: 2025

Recent advancements in large language models (LLMs) have demonstrated extraordinary comprehension capabilities with remarkable breakthroughs on various vision-language tasks. However, the application of LLMs in generating reliable medical diagnostic reports remains in the early stages. Currently, medical LLMs typically feature a passive interaction model where doctors respond to patient queries with little or no involvement in analyzing medical images. In contrast, some ChatBots simply respond to predefined queries based on visual inputs, lacking interactive dialogue or consideration of medical history. As such, there is a gap between LLM-generated patient-ChatBot interactions and those occurring in actual patient-doctor consultations. To bridge this gap, we develop an LLM-based dialogue system, namely proactive multi-round vision-language interactions for computer-aided diagnosis (ProMRVL-CAD), to generate patient-friendly disease diagnostic reports. The proposed ProMRVL-CAD system allows proactive dialogue to provide patients with constant and reliable medical access via an integration of knowledge graph into a recommendation system. Specifically, we devise two generators: a Proactive Question Generator (Pro-Q Gen) to generate proactive questions that guide the diagnostic procedure and a Multi-Vision Patient-Text Diagnostic Report Generator (MVP-DR Gen) to produce high-quality diagnostic reports. Evaluating two real-world publicly available datasets, MIMIC-CXR and IU-Xray, our model has better quality in generating medical reports. We further demonstrate the performance of ProMRVL achieves robust under the scenarios with low image quality. Moreover, we have created a synthetic medical dialogue dataset that simulates proactive diagnostic interactions between patients and doctors, serving as a valuable resource for training LLM.
Network evasion detection with Bi-LSTM model
Venue: arXiv cs.CR 2025
Year: 2025

Network evasion detection aims to distinguish whether the network flow comes from link layer exists network evasion threat, which is a means to disguise the data traffic on detection system by confusing the signature. Since the previous research works has all sorts of frauds, we propose a architecture with deep learning network to handle this problem. In this paper, we extract the critical information as key features from data frame and also specifically propose to use bidirectional long short-term memory (Bi-LSTM) neural network which shows an outstanding performance to trace the serial information, to encode both the past and future trait on the network flows. Furthermore we introduce a classifier named Softmax at the bottom of Bi-LSTM, holding a character to select the correct class. All experiments results shows that we can achieve a significant performance with a deep Bi-LSTM in network evasion detection and it's average accuracy reaches 96.1%.
On Self-Adaptive Perception Loss Function for Sequential Lossy Compression
Venue: arXiv cs.LG 2025
Year: 2025

We consider causal, low-latency, sequential lossy compression, with mean squared-error (MSE) as the distortion loss, and a perception loss function (PLF) to enhance the realism of reconstructions. As the main contribution, we propose and analyze a new PLF that considers the joint distribution between the current source frame and the previous reconstructions. We establish the theoretical rate-distortion-perception function for first-order Markov sources and analyze the Gaussian model in detail. From a qualitative perspective, the proposed metric can simultaneously avoid the error-permanence phenomenon and also better exploit the temporal correlation between high-quality reconstructions. The proposed metric is referred to as self-adaptive perception loss function (PLF-SA), as its behavior adapts to the quality of reconstructed frames. We provide a detailed comparison of the proposed perception loss function with previous approaches through both information theoretic analysis as well as experiments involving moving MNIST and UVG datasets.
BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop
Venue: arXiv cs.CL 2025
Year: 2025

BabyLM aims to dissolve the boundaries between cognitive modeling and language modeling. We call for both workshop papers and for researchers to join the 3rd BabyLM competition. As in previous years, we call for participants in the data-efficient pretraining challenge in the general track. This year, we also offer a new track: INTERACTION. This new track encourages interactive behavior, learning from a teacher, and adapting the teaching material to the student. We also call for papers outside the competition in any relevant areas. These include training efficiency, cognitively plausible research, weak model evaluation, and more.
LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization
Venue: arXiv cs.LG 2025
Year: 2025

We introduce LLM-Lasso, a novel framework that leverages large language models (LLMs) to guide feature selection in Lasso $\ell_1$ regression. Unlike traditional methods that rely solely on numerical data, LLM-Lasso incorporates domain-specific knowledge extracted from natural language, enhanced through a retrieval-augmented generation (RAG) pipeline, to seamlessly integrate data-driven modeling with contextual insights. Specifically, the LLM generates penalty factors for each feature, which are converted into weights for the Lasso penalty using a simple, tunable model. Features identified as more relevant by the LLM receive lower penalties, increasing their likelihood of being retained in the final model, while less relevant features are assigned higher penalties, reducing their influence. Importantly, LLM-Lasso has an internal validation step that determines how much to trust the contextual knowledge in our prediction pipeline. Hence it addresses key challenges in robustness, making it suitable for mitigating potential inaccuracies or hallucinations from the LLM. In various biomedical case studies, LLM-Lasso outperforms standard Lasso and existing feature selection baselines, all while ensuring the LLM operates without prior access to the datasets. To our knowledge, this is the first approach to effectively integrate conventional feature selection techniques directly with LLM-based domain-specific reasoning.