# Retrieval-Augmented Generation to Generate Knowledge Assets and Creation of Action Drivers
## 8 Experiments, Results, and Evaluation  
The RAG TRIAD framework, consisting of Context Relevance, Groundedness or Faithfulness, and Answer Relevance, can be used to evaluate the RAG system to assess the core components as per Table 3.  
There are multiple metrics and tools to measure the RAG system; here, we use the library/system RAGAS [5] (Retrieval-Augmented Generation assessment) to evaluate our RAG model and compare the different architectures. The most important stage of building the RAG system was preprocessing (see Table 4) the data since context and semantics are very important for any tabular data to be used in the RAG system. RAG systems have usually been used in text document fetching, and most methods rely on chunking the texts into different documents. In doing so, context and semantics are lost in the RAG system. Therefore, to compare between various sections in tabular data, a lot of preprocessing has to be performed so the semantic and structural information is not lost in the indexing/chunking process. In this particular case, the sheet data with different comcils have to be preprocessed thoroughly to make a more holistic and simpler tabular sheet for our use in RAG. With the advent of large context windows LLMs, it is easier to feed the whole data to the LLMs, but RAG systems are still used for relevancy and accuracy and to make sure that the LLMs do not hallucinate. The following metrics measure the different components of the architecture. 1. Retriever Component: Context Precision, Context Recall; \begin{table}
\begin{tabular}{l l l l} \hline \hline
**No.** & **Question/Query** & **Naive RAG Answer/Response** & **RAG (With Preprocessing Using LlamaParse) Retrieval k = 7** \\ \hline
1 & Tell me all the locations mentioned in the data are mentioned in this data? & The location mentioned in the data are the locations mentioned in the data are LCR, Wirral, Liverpool, Knowsley, St Helens, Halton, and Sefton. \\ \hline
2 & What are the total companies and scale-ups in Sefton? & In Sefton, there are a total of 2095 companies. Among these, 793 companies and 32 visible scale-ups. & In Sefton, there are a total of 2095 companies. Among these, 793 companies and 81 visible scale-ups. \\ \hline
3 & What are the total companies and scale-ups in Liverpool? & Liverpool has a total of 1833 companies and 72 visible scale-ups. & In Liverpool, there are a total of 5142 companies. Among these, 793 active size-ups. \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Table comparing queries and responses to different models. This shows how preprocessing improves the retrieval and accuracy of the retrieval model. \begin{table}
\begin{tabular}{l l} \hline \hline
**Component** & **Metrics** \\ \hline \multirow{2}{*}{Context Relevance} & Precision, Recall, \\  & MRR (Mean Reciprocal Rank), MAP (Mean Average) \\ \hline Faithfulness & Factual consistency \\ \hline Answer Relevance & BLEU, ROUGE, embedding similarity \\ \hline \hline \end{tabular}
\end{table} 
Table 3: TRIAD framework: component metrics and data mapping. 2. Generator Component: Faithfulness, Answer Relevancy. \[\text{Context Precision}=\frac{\text{Number of relevant contexts retrieved}}{\text{Total number of contexts retrieved}}\]  
\[\text{Context Recall}=\frac{\text{Number of relevant contexts retrieved}}{\text{Total number of relevant contexts available}}\]  
\[\text{Faithfulness}=\frac{\text{Number of claims supported by context}}{\text{Total number of claims in the answer}}\]  
\[\text{Answer Relevancy}=\frac{\text{Relevance score of answer to query}}{\text{Maximum possible relevance score}}\]  
Context Precision is a metric for evaluating Retrieval-Augmented Generation (RAG) systems that measures the proportion of relevant chunks in retrieved contexts. It calculates the mean precision@k across all retrieved chunks, where precision@k represents the ratio of relevant chunks at rank k to the total number of chunks at that rank.