# Enhanced Retrieval-Augmented Generation Using Low-Rank Adaptation
## 3 Proposed Method
### Enhanced Retrieval Strategy
#### 3.2.2 Multi-Stage Filtering Mechanism  
As shown in Figure 2, our retrieval process follows a multi-stage filtering mechanism to refine the document's selection of relevant road traffic-related laws and regulations. This document is stored in embedding vectors from Figure 1 to serve as the foundation for this retrieval process, ensuring an efficient and accurate search.  
**Initial FAISS-based retrieval**: The system retrieves the most relevant top-\(k\) document embeddings vector from the FAISS vector database based on cosine similarity [26] with the query embedding.  
**MMR re-ranking**: The retrieved documents are re-ranked using maximal marginal relevance (MMR) [4]:  
\[\text{MMR}(d_{i})=\lambda\cdot\text{Sim}(q,d_{i})-(1-\lambda)\cdot\max_{d_{i} \in S}\text{Sim}(d_{i},d_{j}) \tag{4}\]  
where \(d_{i}\) is a candidate document to be evaluated, \(q\) is the embedding of the query, \(d_{i}\) is the embedding of the candidate document, \(S\) is the set of already selected documents, and \(\lambda\in[0,1]\) is a balancing parameter that controls the trade-off between relevance and diversity.  
Higher values of \(\lambda\) emphasize selecting documents with high similarity to the query, thereby prioritizing relevance. Lower values favor diversity by penalizing documents that are similar to those already selected. In our experiments, we empirically set \(\lambda=0.7\), which achieved the best balance between retrieving legally relevant content and avoiding redundancy among retrieved clauses.  
**Transformer attention-based filtering and context consolidation**: Transformer attention scores [8] from the LoRA-enhanced BERT-based uncased model further refine the selection. This stage ensures that only the most contextually relevant road traffic-related laws and regulations are retained in the final retrieved set.  
To fully optimize retrieval for road traffic legal applications, the filtered road traffic-related laws and regulations are structured into a consolidated legal context. This involves selecting key road traffic legal provisions and relevant sections that directly address the query intent while eliminating redundant or overly broad content. The extracted road traffic legal information is then organized into a structured format, with Ref. [27] ensuring compatibility with the LLM's prompt input.  
As illustrated in Figure 2, the final structured road traffic legal context is injected into the prompt during the query-answer generation phase. By refining both retrieval and context construction within a unified multi-stage filtering mechanism, our approach enhances road  
Figure 2: Retrieval-augmented road traffic-related laws and regulations processing pipeline. This figure illustrates the retrieval and query–answer generation process. Road traffic-related laws and regulations documents are retrieved using transformer attention-based similarity adjustment and multi-stage filtering (Section 3.2) to construct a structured road traffic legal context. The retrieved context is injected into the prompt, enabling the LLM to generate legally grounded responses (Section 3.3).  
traffic legal accuracy and contextual relevance, ensuring that the LLM generates responses firmly grounded in authoritative road traffic-related laws and regulations document.  
By integrating LoRA-enhanced attention-based retrieval with multi-stage filtering mechanism, our approach substantially outperforms conventional RAG-based systems in terms of retrieval accuracy and contextual relevance.