Adapter Merge Details
#2
by
polymer
- opened
Merge of EverythingLM-V2-13b QLoRa and OpenOrca-Platypus2-13B.
Which specific adapter was merged into the original model? I'd assumed it to be https://huggingface.co/totally-not-an-llm/EverythingLM-13b-V2-peft, but found the max_position_embeddings had been left at 4096. Any details for this (potentially unreleased) adapter?
You are correct, it is https://huggingface.co/totally-not-an-llm/EverythingLM-13b-V2-peft. EveythingLM was trained at normal context length, but https://huggingface.co/totally-not-an-llm/EverythingLM-13b-V2-16k supports 16k context because it is merged with https://huggingface.co/conceptofmind/LLongMA-2-13b-16k instead of the normal llama base model.