NUST Alpha at RIRAG 2025: Fusion RAG for Bridging Lexical and Semantic Retrieval and Question Answering (2025)

Abstract: NUST Alpha participates in the Regulatory Information Retrieval and Answer Generation (RIRAG) shared task. We propose FusionRAG that combines OpenAI embeddings, BM25, FAISS, and Rank-Fusion to improve information retrieval and answer generation. We also explores multiple variants of our model to assess the impact of each component in overall performance. FusionRAG strength comes from our rank fusion and filter strategy. Rank fusion integrates semantic and lexical relevance scores to optimize retrieval accuracy and result diversity, and Filter mechanism remove irrelevant passages before answer generation. Our experiments demonstrate that FusionRAG offers a robust and scalable solution for automating the analysis of regulatory documents, improving compliance efficiency, and mitigating associated risks. We further conduct an error analysis to explore the limitations of our model’s performance.

## introduction the rirag shared task advances question answering (qa) by challenging teams to develop models for accurate query responses over complex regulatory datasets. our team aim to tackle key challenges in retrieval and reasoning while addressing limitations in existing techniques. despite advancements in information retrieval @xcite and answer generation, regulatory information remains underexplored. research has enhanced retrieval-augmented generation (rag) systems using tools like faiss for efficient high-dimensional searches @xcite @xcite , mi-ragdb for gene regulation @xcite , and neo4j for modeling complex relationships in domains like social networks and recommendation systems @xcite @xcite . dense retrieval models like contriever @xcite excel in semantic understanding but struggle with exact keyword matching, while sparse models like bm25 handle lexical matching well but falter with ambiguous queries @xcite . re-ranking methods, such as cross encoders, enhance contextual relevance, and innovations like hyde enrich query generation for ambiguous inputs @xcite . training strategies, like incorporating irrelevant documents, reduce bias and improve robustness. adapter layers, such as linear adapters, fine-tune embeddings for task-specific precision in rag @xcite @xcite , though methods like react (reason + act) show limited industrial applicability @xcite @xcite . the regulatory domain poses challenges due to complex compliance, evolving laws, and regional standards. fusionrag addresses this by combining dense (faiss) and sparse (bm25) retrieval models for nuanced text handling. integrated with chatgpt-3.5, it generates contextually relevant responses tailored to regulatory queries. ## fusionrag figure 1 illustrates our model foot_0 , which integrates vector-based (faiss) and text-based (bm25) retrieval methods to retrieve the most relevant passages. we use a custom rank fusion technique, combining faiss for semantic relevance and bm25 for lexical matching, enhancing retrieval accuracy and diversity. an llm-based prompt (gpt3.5 turbo (openai, 2023)) filters the top-k passages, from which gpt3.5 turbo generates contextually accurate answers, ensuring reliable responses for regulatory queries. ## experiments 3.1 dataset a comprehensive overview of the shared task and dataset are presented to appendix c for brevity. ## conclusions we present fusionrag for the regulatory information retrieval and answer generation (rirag) shared task, combining openai embeddings,smalluppercasebm25, faiss, and rank-fusion (rf) to improve both retrieval and answer generation. our rank fusion strategy merges semantic and lexical relevance scores to enhance accuracy and diversity. we filter top-ranked passages to remove irrelevant results before generating answers. while fusionrag achieves notable improvements in regulatory document analysis, the repass score for generation (0.52) is slightly lower due to a focus on relevance, which impacted entailment and obligation coverage.