# Enhanced Retrieval-Augmented Generation Using Low-Rank Adaptation
## 5 Discussion  
Unlike prior work such as \"Enhancing Retrieval-Augmented Generation: A Study of Best Practices\" [29], which primarily focuses on optimizing retrieval module parameters--such as prompt engineering, chunk size, and retrieval stride--our approach introduces a modular and interpretable re-ranking layer based on MMR. This enables a tunable relevance-diversity trade-off via a single hyperparameter \(\lambda\), and results in computationally efficient, scalable retrieval. Moreover, we apply LoRA to BERT's attention matrices, achieving lightweight fine-tuning suitable for domain-specific legal retrieval tasks--an aspect not explored in the compared best-practices benchmarks. Speculative RAG [17] introduces a novel two-stage generation framework where multiple draft responses are generated using a small specialist language model, and then evaluated by a larger generalist model to select the most coherent and informative final output. This approach mitigates the latency and input length bottlenecks common in standard RAG systems by reducing the number of documents each model processes and leveraging parallel draft generation. It also employs probabilistic measures such as self-containment and self-reflection to filter and refine the generated rationales, resulting in high-quality responses with significantly reduced training time. While speculative RAG offers strong performance in latency-sensitive tasks, its reliance on two distinct language models and rationale evaluation modules introduces architectural complexity and potential resource overhead. In contrast, our proposed framework emphasizes modular efficiency by combining parameter-efficient fine-tuning through LoRA with MMR-based re-ranking. This design allows for a single unified retriever-generator pipeline that is lightweight and interpretable. Specifically, our use of LoRA enables efficient domain adaptation with only 0.54% of the model's parameters updated, while MMR introduces explicit control over relevance-diversity trade-offs through a tunable hyperparameter \(\lambda\). Although this study focuses on LoRA as a parameter-efficient fine-tuning method, we acknowledge that other techniques such as adapter tuning [30] and prompt tuning [31] have also been proposed to reduce fine-tuning costs. Compared to these methods, LoRA offers distinct advantages: it modifies internal attention weights directly without adding new training-time components and retains the original model architecture. Adapter tuning, while modular, often increases training latency due to added adapter layers, and prompt tuning, though lightweight, tends to underperform in tasks requiring deep semantic understanding, such as road traffic legal clause retrieval. To substantiate the parameter-efficiency of LoRA, we conduct a comparative evaluation against adapter tuning, prompt tuning, and full fine-tuning. As shown in Table 5, LoRA achieves a favorable trade-off between performance and efficiency, offering competitive accuracy with significantly fewer trainable parameters. Specifically, it attains 95.83% accuracy with only 0.54% of the parameters updated--substantially lower than full fine-tuning and more efficient than adapter tuning. Empirical benchmarks [5] have shown that LoRA achieves a comparable or superior performance across various downstream tasks while updating fewer parameters. These properties make it particularly suitable for domain-specific applications where both computational efficiency and representational precision are required. Unlike speculative RAG, our method does not require multiple forward passes or ensemble-level evaluation. This leads to lower implementation complexity and greater deployability in constrained road traffic legal or regulatory environments, where transparency and reproducibility are critical. The proposed LoRA-enhanced BERT-based uncased RAG framework substantially improves road traffic-related law and regulation retrieval accuracy and efficiency compared to traditional methods.