README.md

---
base_model:
- meta-llama/Meta-Llama-3-8B
- nvidia/Llama3-ChatQA-1.5-8B
- winglian/llama-3-8b-256k-PoSE
library_name: transformers
tags:
- mergekit
- peft
- nvidia
- chatqa-1.5
- chatqa
- llama-3
- pytorch
license: llama3
language:
- en
pipeline_tag: text-generation
---

# Llama3-ChatQA-1.5-8B-256K

I tried to achive long context RAG pipeline with this model but I have very limited resources to test this workflow. Keep in mind that this is an experimentation.

This model is an 'amalgamation' of `winglian/llama-3-8b-256k-PoSE` and `nvidia/Llama3-ChatQA-1.5-8B`. 

## Recipe

First I extracted the Lora adapter from `nvidia/Llama3-ChatQA-1.5-8B` using `mergekkit`. You can find the adapter [here](https://huggingface.co/beratcmn/Llama3-ChatQA-1.5-8B-lora).

After the extraction I merged the adapter with the `winglian/llama-3-8b-256k-PoSE` model. 

## Prompt Format

Since base model wasn't finetuned for any specific format we can use the ChatQA's chat format.

```text
System: {System}

{Context}

User: {Question}

Assistant: {Response}

User: {Question}

Assistant:
```

Big thanks to Meta Team, Nvidia Team and of course Wing Lian.

## Notes

This model has not been tested on any benchmarks due to compute limitations. Base model wasn't evaluated using `Needle in Haystack` as well. There is a big possibility that this model might perform worse than both of the original models.