# Enhanced Retrieval-Augmented Generation Using Low-Rank Adaptation
###### Abstract
To improve the retrieval and processing of road traffic-related laws and regulations, it is essential to address key challenges inherent in conventional retrieval and representation methods. Traditional retrieval methods suffer from multiple limitations, including issues with semantic representation, contextual loss, and relevance assessment. One key issue is the difficulty in representing the semantic meaning of road traffic-related laws and regulation texts with traditional embedding techniques. Pre-trained language models often fail to capture the subtle nuances and specialized terminology found in road traffic-related laws and regulations, such as those in the \"Road Traffic Act\". This incomplete semantic representation makes it difficult to distinguish between highly relevant and less relevant road traffic-related laws and regulation information. Another limitation is the loss of context during document preprocessing and chunking. Standard preprocessing techniques segment road traffic-related laws, regulations, and documents into fixed token units, which risks fragmenting critical road traffic legal clauses or disrupting contextual continuity. This context loss reduces the precision and reliability of retrieved road traffic-related laws and regulation information. Lastly, traditional retrieval strategies often fail to accurately assess relevance. Conventional retrieval methods, such as FAISS-based vector search combined with maximal marginal relevance (MMR) [4] and similarity score thresholding, may not adequately account for the unique characteristics of road traffic-related laws and regulations. As a result, these methods often retrieve outdated or irrelevant information that does not reflect the latest traffic regulations or regional specifics, potentially leading to flawed decision making in unmanned systems. Given these challenges and gaps in existing approaches, the main objectives of this research are to enhance the semantic representation capabilities of retrieval models for road traffic-related laws and regulations; preserve contextual continuity during document processing; and improve the accuracy and relevance of retrieved road traffic legal information. In this paper, we propose a low-rank adaptation (LoRA) [5]-enhanced RAG framework to improve the accuracy and efficiency of road traffic-related law and regulation retrieval. This framework addresses the above challenges through an optimized retrieval mechanism. Our method builds on the established evidence of LoRA effectiveness in adapting language models while using significantly fewer parameters. Prior research has demonstrated that LoRA can considerably reduce the parameter count while achieving good performance. For BERT-based models specifically, studies have shown that LoRA adaptations can improve domain-specific retrieval capabilities while reducing computational requirements [6]. The proposed approach consists of three key components:  
First, we introduce a LoRA-enhanced bidirectional encoder representations from transformers BERT-based uncased retriever. We refine the retrieval process by applying LoRA to the upper transformer layers, improving the model's ability to capture complex road traffic-related laws and regulation information while reducing computational overhead. Second, we employ a multi-stage filtering mechanism. The retrieval system leverages FAISS-based vector search to obtain relevant road traffic-related laws and regulations and refines the results through a combination of MMR and transformer attention-based filtering [7]. Third, we integrate context-augmented road traffic law text generation. Those retrieved documents are integrated into the prompt of a language model to ensure that generated responses adhere to road traffic-related laws and regulations and road traffic legal standards. By combining these components, the proposed method enhances road traffic-related laws and regulations retrieval, supporting more reliable decision making in unmanned systems.