# Leveraging Retrieval-Augmented Generation for Swahili Language Conversation Systems
## 4 Experimentation
### Experimental Results
This comparison underlines the advantages of FAISS in delivering accurate, contextually relevant, and semantically rich responses, making it the most effective retrieval mechanism for Swahili NLP in the RAG framework. Table 6 highlights the differences in performance across retrieval mechanisms and models. FAISS and Hybrid Retrieval consistently achieved the highest retrieval accuracy and relevance scores across all the models. For example, mT5 with Hybrid Retrieval achieved a retrieval accuracy of 92.15 percent, outperforming the same model with BM25 at 88.50 percent and TF-IDF at 84.45 percent. Response fluency scores followed a similar trend, with Hybrid Retrieval producing the most natural responses for all the models. For instance, GPT-Neo with Hybrid Retrieval achieved a response fluency score of 88.34 percent, which was higher than its performance with FAISS at 86.23 percent and TF-IDF at 80.34 percent. Relevance scores indicate how well the generated responses align with the query's context. Hybrid Retrieval led in relevance, as demonstrated by mBART paired with Hybrid Retrieval achieving a relevance score of 89.34 percent, surpassing its performance with BM25 at 84.78 percent and TF-IDF at 81.45 percent. In terms of efficiency, TF-IDF demonstrated the fastest response times due to its computational simplicity, achieving a low inference time of 0.92 s with GPT-2. However, this speed comes at the cost of lower accuracy and relevance, highlighting the trade-offs between computational efficiency and model performance. The results show that Hybrid Retrieval is the most effective mechanism for Swahili NLP tasks, providing the highest scores in most metrics, but FAISS offers a strong alternative for cases where efficiency is prioritized. The trends also reveal that mT5 consistently performs better across all retrieval mechanisms compared to the other models, making it a strong candidate for Swahili conversational systems. These findings provide valuable insights for selecting the best retrieval mechanism and model combination based on specific application needs and resource constraints. The results in Table 7 compare the diversity, redundancy, and query overlap metrics for various retrieval models in Swahili NLP tasks. Hybrid Retrieval achieved the highest percentage of unique chunks retrieved, at 83.56 percent, reflecting its ability to provide more diverse and contextually rich responses compared to other models. In contrast, TF-IDF had the lowest diversity, retrieving only 68.45 percent unique chunks, which indicates a greater reliance on repetitive or less varied content. This limitation may hinder its effectiveness in addressing complex or nuanced queries. Redundant chunks, which represent repeated or semantically identical content, were minimized in Hybrid Retrieval at 8.67 percent, showcasing its superior capability in filtering out irrelevant repetitions. Conversely, TF-IDF exhibited the highest redundancy rate of 22.34 percent, followed by BM25 at 18.89 percent. These results highlight the importance of advanced retrieval mechanisms in improving content diversity and reducing redundancy for more effective responses. Query overlap measures the alignment between retrieved chunks and the semantic content of the input query. Hybrid Retrieval led with an overlap of 89.45 percent, indicating that it retrieved the most relevant and contextually appropriate information [50]. FAISS followed closely with 85.67 percent overlap, while BM25 and TF-IDF trailed behind, showing their relatively lower alignment with the query intent. In terms of retrieval speed, TF-IDF was the fastest with an average retrieval time of 92 milliseconds, making it a suitable choice for scenarios