An auto-regressive causal LM created by combining 2x finetuned Llama-2 70B into one.
- GGUF (llama.cpp)
- GPTQ (KoboldAI, TGW, Aphrodite)
- AWQ (TGW, Aphrodite, vLLM)
- Exllamav2 (TGW, KoboldAI)
Both Vicuna and Alpaca will work, but due the initial and final layers belonging primarily to Xwin, I expect Vicuna to work the best.
The layer ranges used are as follows:
- range 0, 16 Xwin - range 8, 24 Euryale - range 17, 32 Xwin - range 25, 40 Euryale - range 33, 48 Xwin - range 41, 56 Euryale - range 49, 64 Xwin - range 57, 72 Euryale - range 65, 80 Xwin
Special thanks to @Undi95 for helping with the merge ratios.
- Downloads last month