Enhancing Regulatory Compliance Through Automated Retrieval, Reranking, and Answer Generation (2025)

Abstract: This paper explains a Retrieval-Augmented Generation (RAG) pipeline that optimizes reg- ularity compliance using a combination of em- bedding models (i.e. bge-m3, jina-embeddings- v3, e5-large-v2) with reranker (i.e. bge- reranker-v2-m3). To efficiently process long context passages, we introduce context aware chunking method. By using the RePASS met- ric, we ensure comprehensive coverage of obli- gations and minimizes contradictions, thereby setting a new benchmark for RAG-based regu- latory compliance systems. The experimen- tal results show that our best configuration achieves a score of 0.79 in Recall@10 and 0.66 in MAP@10 with LLaMA-3.1-8B model for answer generation.

## introduction regulatory documents are critical components for many industries including finance, healthcare and insurance, to comply with standards and laws. these documents are characterized by complex legal terminology, hierarchical structures, and frequent updates. therefore, this creates difficulties for interpretation and implementation. these incompatibilities lead to negative outcomes such as significant financial penalties, loss of reputation, and operational disruptions. the complexity of regulatory documents to put forward the necessity for advanced systems capable of efficient information retrieval and synthesis. retrieval-augmented generation (rag) systems offers a promising solution for retrieval mechanism and answer generation. previous research in regulatory natural language processing (regnlp) discovered the poten-* these authors contributed equally to this work tial of machine learning for automating regulatory compliance, but some difficulties still exist: in this study, to address these challenges, we propose an optimized rag pipeline for advanced-level ranking and improved generative performance, using a context-aware chunking strategy combined with "bge-m3 + hybrid search", and "bge-reranker-v2". our contributions are as follows: ## related work the significant progress on regnlp mostly about complexities of regulatory texts. the structured data extraction is focused by the previous studies. in this context, @xcite focus on xmlbased frameworks in order to extract information from accessibility regulations. also, @xcite propose the cerno framework to focus on automation of rights and obligation extraction from legal texts. however, these works are insufficient in scalability and adaptability. thanks to the advent of deep learning, considerable improvements achieved in regnlp. in this context, @xcite introduce a hierarchical bilstm model in order to extract obligations from legal contracts, and this study outperforms the previous methods that relies on manual features. in addition, @xcite implement deep learning pipelines in the work of annotating global trade regulations. this method enables enhanced compliance workflows in the field of reg-nlp. similar to these works, @xcite leverage bert-based models to handle complex queries in eu/uk legislative texts. this method shows how transformer architecture is effective in processing long documents. @xcite extend this method with bert for automated question-answering (qa) systems targeting gdpr-related texts. thanks to this work, a considerable success has achieved in passage retrieval tasks. @xcite provide a baseline framework for regulatory qa tasks by introducing the obliqa dataset foot_0 curated to address multi-passage queries. this dataset is as collection of over 27,000 qa pairs derived from abu dhabi global markets foot_1 regulations. additionally, this study introduces regulatory passage answer stability score (repass), a novel evaluation metric designed to measure the accuracy and consistency of generated answers in regulatory contexts. they combine sparse and dense retrieval methods (e.g., bm25 and bge models) with a generative approach to synthesize answers from retrieved passages. despite the contribution of this work, they challenged in handling complex or lengthy queries, and the generative model exhibited limitations in contextual comprehension and obligation coverage. regnlp applications accelerated by the recent advancements in synthetic data generation. an example of these upgrades is qa dataset for roundtrip validation in @xcite . also, @xcite propose zero-shot learning method for neural passage retrieval. the integration of retrieval and generative models enables advanced qa methodologies in rag systems. @xcite formalize rag as a framework that enriches generative models with retrieved knowledge. the retrieval efficiency and response quality is improved by self-rag @xcite ) and piperag ( jiang et al. (2024) ) systems having limited adaptation to regulatory texts. our study addresses challenges in retrieval precision and generative accuracy for regulatory qa. we introduce a robust rag pipeline incorporating hybrid retrieval using dense models, advance re-ranking and context-aware chunking to manage long regulatory documents. this system achieves a recall@10 of 0.79 and map@10 of 0.66, establishing a new standard for regulatory question answering. ## methodology the long passages in regulatory documents affect the performance of generative models, since processing extended contexts efficiently is a challenge for these models. in order to handle this challenge, we segment long passages into smaller chunks, then filter and re-rank to optimize the input for generative models. in the next sections, we describe the proposed methodology by explaining the retrieval pipeline, long-context processing techniques, and answer generation system. the demonstration of our pipeline is shown in figure 1 . ## conclusion this paper proposes rag pipeline for regulatory compliance tasks by integrating hybrid search, advanced re-ranking and context-aware chunking strategies. rag performance improved precision and recall significantly by using bge-m3 model for hybrid search and bge-reranker-v2 model for re-ranker. according to the experimental results, we achieve a score of 0.79 at recall@10 and 0.66 at map@10. introducing the chunk-based processing approach enhanced the llama-3.1-8b-instruct model's generative capabilities and enabled more effective processing of long-context regulatory documents.