---
license: llama2
tags:
- moe
- merge
---
<img src=https://huggingface.co/lodrick-the-lafted/Grafted-Llama2-2x70B/resolve/main/gl.png>
The Llamas are WinterGoddess + AuroraNights.

This is yet another mergekit abomination. 

Still undergoing testing.

This is probably more of a "dense" MoE than a sparse one.