# Context Is King: Large Language Models' Interpretability in Divergent Knowledge Scenarios
## 4 Experimental Setup
### Evaluated Models and Configuration  
The baseline models and pipeline were established using prompt engineering techniques to directly create the queries and integrate the models through APIs provided by OpenAI (for GPT models) and Amazon Web Services (AWS) Bedrock (for the other models). This approach enables precise control over the prompt construction process, ensuring that the appropriate context from the Divergent Knowledge Dataset is injected into each question and that additional instructions can be included as needed.  
By leveraging the APIs from OpenAI and AWS Bedrock, we ensure seamless integration of different LLMs without requiring modifications to the rest of the pipeline. This flexibility facilitates the dynamic construction of contextually enriched prompts, guiding the models to generate accurate and relevant responses. In this work, several state-of-the-art models were evaluated using this setup, ensuring a diverse representation of model architectures and capabilities. The models evaluated include the following:  
* GPT-3.5 Turbo [2]: An enhanced version of GPT-3, known for its improved performance and efficiency in generating human-like text. It has 175 billion parameters and is trained on a diverse range of internet text. Inference for GPT-3.5 Turbo was conducted using the OpenAI API.
* GPT-4o [56]: The latest iteration in the GPT series, featuring even more parameters and fine-tuned capabilities. GPT-4o offers superior accuracy and a deeper understanding of context compared to its predecessors. Inference for GPT-4o was conducted using the OpenAI API.
* Llama 3 (70B) [57]: Developed by Meta, this model features 70 billion parameters and is designed for large-scale language tasks. Llama 3 demonstrates state-of-the-art performance on a wide range of industry benchmarks and includes new capabilities such as improved reasoning and code generation. Inference for Llama 3 was conducted using AWS Bedrock.
* Mixtral 8x7B [58]: Developed by Mistral AI, this is a Sparse Mixture-of-Experts (SMoE) model that utilizes only 12.9 billion active parameters out of a total of 46.7 billion. It is designed for efficiency and performance, with strong capabilities in multiple languages and coding tasks. Inference for Mixtral was conducted using AWS Bedrock. It is the smallest model we are testing, aiming to determine if our interpretability thesis holds with less powerful models.  
For all models, the inference configuration included a temperature of 0.2, a top-k value of 50 (used only for Llama 3 and Mixtral), and a top-p of 0.9. A low temperature setting, such as 0.2, reduces randomness in the output, encouraging the model to produce more consistent and focused responses by prioritizing high-probability words. The top-k sampling, set to 50, limits the choice in next words to the 50 most probable options, allowing some variability while preventing unlikely words from being considered. Lastly, a top-p value of 0.9, or nucleus sampling, dynamically narrows the selection to a subset of words whose cumulative probability reaches 90%, balancing precision with slight diversity. This consistent configuration ensures a fair and controlled comparison across different models, optimizing for accuracy and coherence in responses.  
By utilizing these diverse models, we aim to provide a comprehensive evaluation of the CDK-E methodology across different architectures and capabilities. This approach ensures that our findings are robust and generalizable across various state-of-the-art LLMs.