A REGNLP Framework: Developing Retrieval-Augmented Generation for Regulatory Document Analysis (2025)

Abstract: This study presents the development of a Retrieval-Augmented Generation (RAG) framework tailored for analyzing regulatory documents from the Abu Dhabi Global Markets (ADGM). The methodology encompasses comprehensive data preprocessing, including extraction, cleaning, and compression of documents, as well as the organization of the ObliQA dataset. The embedding model is utilized for generating embeddings during the retrieval phase, facilitated by the txtai library for managing embeddings and streamlining testing. The training process incorporated innovative strategies such as duplicate recognition, dropout implementation, pooling adjustments, and label modifications to enhance retrieval performance. Hyperparameter tuning further refined the retrieval component, with improvements validated using the recall@10 metric, which measures the proportion of relevant passages among the top-10 results. The refined retrieval component effectively identifies pertinent passages within regulatory documents, expediting information access and supporting compliance efforts.

## introduction regulatory documents are comprehensive texts that outline mandatory rules and guidelines for organizational compliance. their complexity presents significant challenges in manual analysis, often leading to inefficiencies and errors @xcite . advances in natural language processing (nlp) offer promising solutions to these challenges @xcite @xcite . this study focuses on the development of the retrieval phase of a retrieval-augmented generation (rag) framework, aiming to accurately identify related information within abu dhabi global markets (adgm)'s regulatory documents. by enhancing retrieval accuracy, the framework seeks to facilitate rapid access to relevant information, thereby supporting effective compliance and decision-making processes. the methodology involves comprehensive data preprocessing, including extraction, cleaning, and compression of documents, as well as the organization of the obliqa dataset. the embedding model is selected for embedding generation due to its efficiency in producing high-quality text representations. to streamline testing procedures, the txtai library is utilized, serving as an all-in-one embedding database that supports semantic search and language model workflows. the training process incorporated innovative strategies such as duplicate recognition, dropout implementation, pooling adjustments, and label modifications to enhance model performance. hyperparameter tuning further optimized the retrieval component, and retrieved passages are validated by the recall@10 metric. the refined retrieval framework effectively identifies relevant passages within regulatory documents, accelerating information access and supporting compliance efforts. this study underscores the transformative potential of integrating nlp technologies into regulatory processes, laying a solid foundation for future research aimed at developing comprehensive rag systems. ## related work the application of rag methods to regulatory workflows remains an underexplored area in the literature. @xcite highlights the potential of rag to improve efficiency in distinct domains by combining information retrieval and generation. however, the study notes that the implications of rag for regulatory documents require further in-vestigation. the integration of nlp into regulatory processes has been explored across various sectors, including the construction industry, financial, and healthcare sectors. in the construction industry, zhang and el-gohary (2016) utilize semantic-based information extraction to automate compliance checks within construction regulations, reducing manual effort and expediting processes. in the financial sector, @xcite develops nlp tools to analyze financial regulatory documents, enhancing both accuracy and operational efficiency. in the healthcare sector, @xcite employ bert-based models to classify potential risks in drug labeling texts, providing rapid analyses for regulatory agencies. subsequently, @xcite introduces rxbert, improving information extraction from drug labeling documents. ## dataset the dataset comprises 40 regulatory documents provided by adgm, each ranging from approximately 30 to 100 pages. these documents are segmented into passages, with each passage stored as a json file containing "id," "documentid," and "passageid." the passages average 60 words, with lengths varying from 1 to 24,312 words. the test dataset includes 2,786 questions, each accompanied by "questionid," "question," and the corresponding passages expected to be retrieved. ## methodology the methodology encompasses data preprocessing, model selection and training procedures. ## results and discussion recall@10 measures the proportion of relevant passages among the top 10 returned results. the models are evaluated using the recall@10 metric. prototype 1 achieved a recall@10 of 0.76, while prototype 2 improved to 0.78, indicating enhanced retrieval effectiveness. these results mean that duplicate detection, the addition of dropout, pooling adjustments, and adjustments in the labels are responsible for prototype 2's higher performance. the higher recall@10 score shows that the model is better at correctly selecting relevant passages within the regulatory documents. however, the incremental improvement from one prototype to another indicate that further improvements are needed to achieve even more significant retrieval performances. future work should proceed in the direction of exploiting further training techniques, refining hyperparameters, and using more complex models to further improve the model's performance in processing complex regulatory texts. ## conclusion the integration of nlp into regulatory processes has huge potential to facilitate compliance efficiency in many industries. this paper contributes to this dynamic area by developing a rag model focused on the analytical aspects of regulatory documents obtained from the adgm. the focus of the paper on the retrieval component of the rag model enables the study to address certain challenges related to the extraction of relevant information in long and complex regulatory texts. the methodology includes detailed data preprocessing that enables document extraction and cleaning to ensure the quality and relevance of the dataset. the choice of the model to generate embeddings, combined with the work using the txtai library, allowed fast embedding and smooth testing. training included state-of-the-art methods, such as duplicate detection, dropout, and tuning pooling for better performance. this model is further optimized by applying techniques for hyperparameter tuning; the retrieval accuracy is improved, as estimated from the recall@10 metric. these results confirm that the refined retrieval model efficiently retrieves relevant regulatory passages to speed up access and compliance to information. this is particularly important, given the complexity and volume of regulatory texts, usually beyond manual human analysis. in conclusion, this work points out the transformational role that nlp technologies, in particular rag frameworks, could play if embedded in regulatory processes. realized progress during the retrieval phase provides a firm base for subsequent research to build up general rag systems. further work will have to be addressed for the development of more robust generation techniques and fine-tuned embedding models, which can allow improving the overall compliance workflows. such systems have the potential to revolutionize regulatory compliance by providing accurate and contextually relevant information, leading to an agile and responsive regulatory environment.