# Leveraging Retrieval-Augmented Generation for Swahili Language Conversation Systems
## 4 Experimentation
### Experiment Settings  
The experiments were conducted to evaluate the integration of retrieval and generation within the Retrieval-Augmented Generation framework for Swahili Natural Language Processing. Four pre-trained models--mT5, GPT-2, mBART, and GPT-Neo--were tested to generate accurate, fluent, and contextually grounded responses to Swahili queries. These models were carefully selected for their strong performance in multilingual and low-resource settings, as they provide diverse architectural advantages for tasks involving Swahili NLP. The pre-trained models, listed in Table 3, were fine-tuned on the curated dataset with an 80-10-10 split for training, validation, and testing. These splits ensured sufficient examples for training while maintaining independent datasets for unbiased evaluation. Additionally, preprocessing steps specific to Swahili were employed to harmonize linguistic variations and enhance data quality, thereby improving the models' effectiveness. Hyperparameter tuning was performed systematically to achieve an optimal balance between computational efficiency and model performance. A batch size of 16 was used \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Dataset Component** & **Train Entries** & **Validation Entries** & **Test Entries** \\ \hline Swahili Queries (Fine-Tuning) & 4500 & 1000 & 1500 \\ Paraphrased Questions (Fine-Tuning) & 1500 & 500 & 500 \\ Noisy Text (Fine-Tuning) & 500 & 250 & 250 \\ \hline External Context Chunks (RAG) & 3000 & – & – \\ \hline Total Entries & 7000 & 1750 & 2250 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Dataset composition for Retrieval-Augmented Generation (RAG) in Swahili NLP. to optimize memory usage while ensuring stable gradient updates. The learning rate was set at \(2\times 10^{-5}\), a value derived from empirical trials to stabilize convergence and minimize overfitting. Training was conducted over five epochs, which provided sufficient coverage of the dataset without introducing redundancy.The experiments were conducted on NVIDIA Tesla V100 GPUs with 32 GB of memory (Nvidia, Santa Clara, CA, USA), as shown in Table 3. This hardware facilitated efficient training of large models such as mT5 and GPT-Neo, which require significant computational resources. Fast SSD storage with a capacity of 2 TB was employed to ensure rapid access to datasets and embeddings during both training and retrieval operations. FAISS was employed as the retrieval backend for vector similarity search, using cosine similarity to measure the relevance of retrieved text chunks. FAISS was chosen for its scalability and efficiency in managing high-dimensional vector data, making it an optimal choice for the Swahili knowledge database. The cosine similarity metric was applied to ensure semantic relevance between query embeddings and document chunks. The experiments evaluated three critical challenges in Swahili NLP: handling rephrased questions, noisy text, and multi-turn dialogues. Rephrased questions tested the models' ability to generalize across variations in query phrasing. Noisy text introduced typographical errors and informal syntax to simulate real-world scenarios. Multi-turn dialogues assessed the models' capacity to maintain context over multiple exchanges. This experimental design ensured a comprehensive evaluation of the models' capabilities and highlighted the effectiveness of the Retrieval-Augmented Generation framework in addressing the unique challenges of low-resource languages like Swahili [45; 46; 47].