# Context Is King: Large Language Models' Interpretability in Divergent Knowledge Scenarios
## 3 CDK-E: Context-Driven Divergent Knowledge Evaluation  
In this section, we introduce the Context-Driven Divergent Knowledge Evaluation (CDK-E) methodology, the principal contribution of our work. CDK-E is designed to evaluate the interpretability of large language models in scenarios that rely on external context rather than the model's internal knowledge. Specifically, the methodology tests LLMs using contexts that diverge from the inherent knowledge, meaning that the models can only generate correct responses based on the provided information. By assessing the accuracy of these context-dependent responses, CDK-E offers a more interpretable and reliable evaluation of the models' decision-making processes. This approach complements other XAI techniques by establishing a framework for leveraging context-based methods to interpret model outputs under controlled knowledge conditions.  
To support the implementation of this methodology, we also introduce the Divergent Knowledge Dataset (DKD), a dataset specifically designed to provide these divergent contexts necessary for testing the approach. It contains fabricated contexts, questions, and ground-truths that challenge the models' inherent knowledge, forcing them to rely on the provided context to produce accurate responses.  
The CDK-E methodology enables evaluating the interpretability of an LLM by obtaining answers to the DKD questions given a specific context, assessing their correctness, and, therefore, evaluating the model's interpretability under controlled knowledge conditions. Figure 1 illustrates a diagram with the CDK-E methodology's key components and process flow.  
In this process, the fabricated contexts and DKD questions are integrated with the prompt in the inference pipeline, forming the query sent to the LLM under evaluation. In this study, we assess multiple models to compare their performance in these divergent-context scenarios. Then, the generated responses are analyzed within the performance assessment module, which utilizes independent evaluator LLMs to compute key performance and interpretability metrics. This module compares the generated answers with ground-truth answers to calculate the metrics, thus providing insights into the model's ability to interpret and adapt to specific contexts while addressing key challenges in XAI for LLMs, such as model reliability, transparency, and alignment with provided contexts.  
Specifically, CDK-E methodology addresses the following:  
* Biases and Hallucinations: By using divergent contexts, CDK-E enhances the detection of hallucinations and biases in LLM responses as these issues become more apparent when the model is exposed to information that conflicts with its internal knowledge.
* Contextual Misalignment: By evaluating LLM outputs against the provided context, CDK-E helps to identify any misalignment between generated and ground-truths.
* Scalability: The model-independent pipeline, combined with prompt engineering, enables scalable and flexible evaluation across various models and datasets.  
In the following sections, the key components are outlined, including dataset fabrication and the performance assessment module.