# Hybrid Multi-Agent GraphRAG for E-Government: Towards a Trustworthy AI Assistant
## Appendix A Appendix
### Faithfulness Assessment  
This subsection presents a comprehensive evaluation of four QA pipelines: Embeddings RAG, GraphRAG, web search, and agentic, based on their factual accuracy (faithfulness) and latency. The assessment process was LLM-based, utilizing GPT-4o to systematically analyze and score the outputs of each pipeline. For each of the 60 test questions, 30 related to \"Clean Industrial Deal\" and the other 30 related to \"US tariffs\" generated answers that were decomposed into factual statements and assessed using a custom-designed faithfulness evaluation framework. This framework examined whether each claim was supported, unsupported, or inconclusive, based on the evidence provided by the respective retrieval system. The following tables summarize key metrics, including statement-level support rates, answer-level faithfulness, and response latency for each pipeline. This analysis provides detailed insight into the relative strengths and trade-offs of each approach, setting the stage for further discussion and interpretation of the results. The evaluation results, as presented in Table 3, highlight distinct differences in faithfulness between Embeddings RAG and GraphRAG, two pipelines that both operate on the same internal dataset, but employ different retrieval strategies. GraphRAG outperforms Embeddings RAG across nearly every metric of factual support: 95.1% of statements generated by GraphRAG were explicitly supported by the retrieved context, compared to 93.6% for Embeddings RAG. This advantage is also evident at the answer level, where 76.7% of GraphRAG responses were fully supported (i.e., every statement grounded in evidence), compared to 68.3% for Embeddings RAG. The rate of unsupported statements is almost halved in GraphRAG (3.9%) relative to Embeddings RAG (5.9%), and the proportion of answers containing at least one unsupported claim drops from 26.7% to 13.3%. These results suggest that the structured, targeted queries of GraphRAG are more effective at surfacing precise and relevant information than the semantic similarity searches of Embeddings RAG. For additional context, the web search pipeline achieved the highest levels of faithfulness among all individual pipelines, with 99.3% of statements supported and 93.3% of answers fully grounded in retrieved evidence. However, it is important to note that web search benefited from access to the full content of live web pages, allowing for retrieval of richer and more extensive contexts than was possible for Embeddings RAG or GraphRAG, which were both limited to briefer internal documents or knowledge graph entries. As \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Metric/Pipeline** & **Embeddings RAG** & **GraphRAG** & **Web Search** & **Agentic** \\ \hline Total Questions Evaluated & 60 & 60 & 60 & 60 \\ \hline Supported Statements (\%) & 93.6\% (671) & 95.1\% (627) & 99.3\% (1012) & 99.7\% (699) \\ \hline Unsupported Statements (\%) & 5.9\% (42) & 3.9\% (26) & 0.7\% (7) & 0.1\% (1) \\ \hline Inconclusive Correct (\%) & 0.6\% (4) & 0.9\% (6) & 0.0\% (0) & 0.1\% (1) \\ \hline Inconclusive Incorrect (\%) & 0.0\% (0) & 0.0\% (0) & 0.0\% (0) & 0.0\% (0) \\ \hline Questions w/Supported & 58 (96.7\%) & 57 (95.0\%) & 60 (100.0\%) & 60 (100.0\%) \\ \hline Questions w/Unsupported & 16 (26.7\%) & 8 (13.3\%) & 4 (6.7\%) & 1 (1.7\%) \\ \hline Questions w/Inconcl. Correct & 3 (5.0\%) & 6 (10.0\%) & 0 (0.0\%) & 1 (1.7\%) \\ \hline Questions w/Inconcl. Incorrect & 0 (0.0\%) & 0 (0.0\%) & 0 (0.0\%) & 0 (0.0\%) \\ \hline Fully Supported Answers & 41 (68.3\%) & 46 (76.7\%) & 56 (93.3\%) & 58 (96.7\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: LLM-based statement and answer level faithfulness metrics across QA pipelines. such, while web search sets a high bar for faithfulness, its broader retrieval scope makes direct comparison with the other two pipelines less straightforward.